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FOREWORD 


The  MURI  project  titled  Characterization  and  Mitigation  of  Failures  in  Complex 
Dynamical  Systems  was  started  on  June  1,  2001  and  ended  on  May  31,  2007.  It  was 
conducted  under  the  leadership  of  Pennsylvania  State  University,  University  Park,  PA 
and  other  participating  universities  were  Carnegie-Mellon,  Louisiana  Tech,  and  Duke. 
This  research  project  has  developed  a  novel  approach  to  anomaly  characterization  and 
real-time  decision-making  in  complex  dynamical  systems  to  achieve  pervasive  fault 
tolerance  based  on  the  emerging  information.  Extensive  experimental  research  has  been 
conducted  to  validate  the  theoretical  results.  Special-purpose  laboratory  apparatuses  have 
been  procured.  On  the  experimental  side  of  the  CSF  MURI  project,  three  laboratories 
have  been  constructed  at  Penn  State  under  equipment  support  from  DURIP  grants  for  five 
consecutive  years. 

The  research  work,  conducted  under  this  research,  has  resulted  in  publication  of  more 
than  125  scholarly  publications,  one  research  monograph,  and  completion  of  15  doctoral 
dissertations.  This  MURI  project  has  effectively  transferred  the  newly  developed 
technology  to  defense  industry  through  five  SBIR  and  STTR  projects  in  collaboration 
with  industry. 
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STATEMENT  OF  THE  PROBLEM  STUDIED 


The  research,  conducted  under  this  MURI  project,  focused  on  characterization  and 
real-time  mitigation  of  pervasive  faults  in  human-engineered  complex  dynamical 
systems.  These  systems,  although  architecturally  similar  to  physical  systems,  are  often 
structurally  quite  different.  At  the  lowest  level  of  decomposition,  the  macroscopic  effects 
could  be  triggered  by  single  fault  manifestations  of  emerging  physical  defects  in 
hardware,  erroneous  states  of  software,  or  human  operator  errors.  Analytical  methods 
have  been  developed  and  experimentally  validated  for  sustaining  order  and  nonnalcy  by 
deriving  and  finding  critical  values  of  physical  parameters,  where  the  subsequent 
behavior  of  the  macroscopic  system  could  change  abruptly. 

For  theoretical  analyses,  complex  dynamical  systems  have  been  modeled  as  hybrid 
interacting  automata,  whose  continuously  varying  dynamics  characterize  the  physical 
process  at  the  lowest  level  of  abstraction.  Discrete  event  models  at  the  higher  levels 
capture  the  cognitive  response  of  the  system  to  observed  emerging  phenomena.  This 
concept  has  been  used  to  utilize  the  dynamical  behavior  of  structural  materials  in 
formulating  damage  mitigating  control  algorithms  at  the  system  level  to  enhance  the  life 
of  critical  mechanical  components.  The  research  has  formulated  analytical  models  of 
component  interactions  triggered  by  different  types  of  failures  to:  (/)  predict  emerging 
pathological  system  behavior  from  time-series  observations  of  events  and  their  dynamic 
interactions,  and  (//)  formulate  adaptive  mechanisms  to  circumvent  or  mitigate  the  effects 
of  pathological  behavior.  The  research  efforts  have  resulted  in  comprehensive 
characterization  of  pathological  behavior,  both  syntactic  and  operational,  through 
analysis  of  spatial-temporal  patterns  in  databases  of  event/action  dynamics  as  well  as  by 
extensive  experimentation. 

The  research  has  provided  a  scientific  basis  for  engineering  dependability  in  military 
operations  with  a  fundamentally  new  approach  to  engineering  and  operation  of  complex 
informational  systems  for  pervasive  fault  tolerance.  Instead  of  specifying  parameters  for 
worst-case  design  of  system  components,  these  systems  are  designed  by  specifying  a 
scalable  set  of  resources  (components)  that  interact  to  support  evolving  operational  needs 
of  defense  applications  in  a  dynamic  and  uncertain  environment. 
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SUMMARY  OF  THE  MOST  IMPORTANT  RESULTS 


Multidisciplinary  University  Research  Initiative  (MURI)  Project 

Characterization  and  Mitigation  of  Failures  in  Complex  Dynamical  Systems 
Principal  Investigator:  Professor  Asok  Ray,  Pennsylvania  State  University 
Participating  Universities:  Carnegie-Mellon;  Louisiana  Tech;  and  Duke 

The  objective  of  the  Complex  Systems  Failure  (CSF)  Multidisciplinary  University 
Research  Initiative  (MURI)  project  has  been  to  fonnulate  and  disseminate  a  knowledge  base 
of  science  and  technology  for  enhancing  dependability  of  complex  engineering  systems  that 
include  both  human-operated  and  unmanned  machinery.  It  has  developed  a  fundamentally 
new  approach  to  anomaly  characterization  and  real-time  decision-making  in  complex 
dynamical  systems  to  achieve  pervasive  fault  tolerance  based  on  the  emerging  information. 
Dependability  of  complex  systems  is  achieved  by  identifying  and  mitigating  the  origins  of 
chaos  and  disorder  at  a  very  early  stage  through  dynamic  coordination  and  control  of  the 
critical  subsystems. 

On  the  theoretical  side  of  the  CSF  MURI  project,  complex  dynamical  systems  (e.g., 
chaotic  motion  and  bifurcation)  are  modeled  from  the  perspectives  of  (Classical  and 
Quantum)  Statistical  Mechanics  and  Automata  Theory  to  retrieve  relevant  information 
for  decision  and  control.  In  this  approach,  dynamical  systems  are  represented  by  hybrid 
interacting  automata  where  continuously  varying  information  can  be  captured  at  various 
levels  of  abstraction.  For  example,  this  concept  utilizes  the  information  on  multi-fractal 
behavior  of  ductile  alloys  in  fonnulating  damage  mitigating  decision  &  control 
algorithms  at  the  system  level  to  enhance  the  service  life  of  critical  electromechanical 
components  in  complex  systems.  The  goal  is  to  formulate  analytical  models  of 
component  interaction  dynamics  triggered  by  various  types  of  exogenous  disturbances 
and  intrinsic  failures  to  (/)  predict  emerging  pathological  behavior  of  the  dynamical 
system  from  observations  of  time-series  data  and  time-dependent  discrete  events;  and  (//) 
formulate  adaptive  mechanisms  to  circumvent  or  mitigate  the  effects  of  pervasive  faults. 

Extensive  experimental  research  has  been  conducted  to  validate  the  above  theoretical 
results.  Special-purpose  laboratory  apparatuses  have  been  procured.  On  the 
experimental  side  of  the  CSF  MURI  project,  the  following  three  laboratories  have  been 
constructed  at  Penn  State  under  equipment  support  from  DURIP  grants  for  five 
consecutive  years: 

•  Electromechanical  and  Electronic  Systems  Laboratory 

•  Networked  Robotic  Systems  and  Signal  Intelligence  Laboratory 

•  Engineering  Systems  Simulation  Laboratory 

The  research  work,  conducted  under  this  MURI  from  June  1,  2001  to  May  31,  2007, 
has  been  reported  in  archive  journals,  refereed  conferences,  patent  disclosures,  and 
research  monographs,  as  listed  below. 
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•  Over  fifty  archive  journal  publications 

•  Seventy  five  refereed  conference  publications 

•  Three  patent  disclosures 

•  One  research  monograph  and  several  book  chapters 

This  MURI  project  has  supported  training  of  five  postdoctoral  scholars  and  has 
produced  fifteen  doctoral  dissertations.  A  majority  of  the  graduate  students,  upon 
completion  of  their  doctoral  education,  have  accepted  research  employment  in  DoD 
laboratories  and  defense  contractors. 

This  MURI  project  has  effectively  transferred  the  newly  developed  technology  to 
defense  industry  through  five  SBIR  and  STTR  projects  in  collaboration  with  industry. 
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74.  I.  Chattopadhyay  and  A.  Ray,  “Generalized  Projections  &  Decidability  of  State  Determinacy  in  Finite 
State  Automata,”  American  Control  Conference,  New  York,  NY,  July  2007. 

75.  R.  P.  Patankar,  V.  Rajagopalan,  D.  Tolani  and  A.  Ray,  “Prognosis  of  Failure  Precursor  in  Complex 
Electrical  Systems  Using  Symbolic  Dynamics,”  American  Control  Conference,  New  York,  NY,  July 
2007. 


(c)  Manuscripts  submitted  but  not  yet  published. 

1.  M.  Yasar,  A.  Ray  and  J.  F.  Horn,  “A  Comprehensive  Control  Strategy  for  Integrated  Flight/Propulsion 
Systems”,  Control  Engineering  Practice,  under  review. 

2.  M.  Yasar,  S.  Sarkar,  S.  Gupta  and  A.  Ray,  “Fault  Diagnosis  and  Isolation  in  Aircraft  Gas  Turbine 
Engines:  Part  I  -  The  Underlying  Concept,”  AIAA  Journal  of  Propulsion  and  Power,  under  review. 

3.  M.  Yasar,  S.  Sarkar,  S.  Gupta  and  A.  Ray,  “Fault  Diagnosis  and  Isolation  in  Aircraft  Gas  Turbine 
Engines:  Part  II  -  Validation  on  a  Simulation  Test  Bed,”  AIAA  Journal  of  Propulsion  and  Power, 
under  review. 

4.  A.  Srivastav,  A.  Ray  and  S.  Gupta,  “An  Information-theoretic  Measure  for  Anomaly  Detection  in 
Complex  Dynamical  Systems,”  Mechanical  Systems  and  Signal  Processing,  under  review. 

5.  C.  Rao,  S.  Sarkar,  M.  Yasar  and  A.  Ray,  “Comparative  Evaluation  of  Symbolic  Dynamic  Filtering  for 
Detection  of  Anomaly  Patterns,”  Pattern  Recognition,  under  review. 

6.  I.  Chattopadhyay  and  A.  Ray,  “Generalized  Projections  in  Finite  State  Automata,”  IEEE  Transactions 
on  Automatic  Control,  under  review. 
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7.  V.  Rajagopalan,  S.  Chakraborty  and  A.  Ray,  “Estimation  of  Slowly- Varying  Parameters  in  Nonlinear 
Systems  via  Symbolic  Dynamic  Filtering,”  Signal  Processing ,  under  review. 

8.  S.  Chakraborty,  S.  Gupta,  A.  Ray  and  A.  Mukhopadhyay,  “Dynamic  Data-Driven  Detection  and 
Estimation  of  Faults  in  Thermal  Pulse  Combustors,”  ASME  Journal  of  Dynamic  Systems, 
Measurement,  and  Control ,  under  review. 

9.  G.  Mallapragada,  I.  Chattopadhyay  and  A.  Ray,”  Automated  Behavior  Recognition  in  Mobile  Robots 
Using  Symbolic  Dynamic  Filtering,”  Robotics  and  Autonomous  Systems ,  under  review. 


(d)  Book  and  book  chapters 

1.  A.  Ray,  V.  Phoha  and  S.  Phoha,  Quantitative  Measure  for  Discrete  Event  Supervisory  Control: 
Theory  and  Applications,  Springer,  New  York,  2005.  ISBN  0-387-02108-1. 

2.  A.  Ray  and  S.  Phoha,  “Sensor  Calibration  and  Estimation  for  Real-Time  Monitoring  and  Control,” 
Frontiers  In  Distributed  Sensor  Networks,  CRC  Press,  Boca  Raton,  Florida,  eds.  S.S.  Iyenger  and  R. 
Brooks,  2005,  pp.  381-398. 

3.  V.V.  Phoha,  S.  Phoha,  A.  Ray,  K.S.  Balagani,  A.U.  Nadgar  and  R.  Varanasi,  “Embedded  Soft  Sensing 
for  Anomaly  Detection  in  Mobile  Robotic  Networks,”  Sensor  Network  Operations,  IEEE  Press/Wiley, 
eds.  S.  Phoha,  T.  LaPorta  and  C.  Griffin,  Chapter  10,  2006,  pp.  609-629. 

4.  S.  Gupta  and  A.  Ray,  “Symbolic  Dynamic  Filtering  for  Data-driven  Pattern  Recognition,”  Pattern 
Recognition  Research  Horizons,  ed.,  F.  Columbus,  Nova  Science  Publishers,  Hauppage,  NY,  in  press. 


(e)  Technical  reports 

1.  First  Year  Interim  Report  for  2001  MUR1  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  March  2002. 

2.  Second  Year  Interim  Report  for  2001  MURI  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  March  2003. 

3.  Third  Year  Interim  Report  for  2001  MURI  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  March  2004. 

4.  Third  Year  Interim  Report  for  2001  MURI  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  August  2004. 

5.  Fourth  Year  Interim  Report  for  2001  MURI  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  August  2005. 

6.  Fifth  Year  Interim  Report  for  2001  MURI  on  Characterization  and  Mitigation  of  Service  Failures  in 
Complex  Dynamic  Systems,  funding  number  C-DAAD19-01-1-0646,  August  2006. 
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REPORT  OF  INVENTIONS 


1.  V.  V.  Phoha,  Sunil  Babu,  Asok  Ray,  Shashi  Phoha,  “System  and  Method  for 
Classifying  Regions  of  Keystroke  Density  with  Neural  Network,”  Technology 
licensed  to  BioPassword  Inc.,  Washington.  Patent  filed  through  Blakely  Sokoloff 
Taylor  &  Zafman,  LLP. 

2.  V.  V.  Phoha,  “Choice  of  Sample  Size  for  Keystroke  Authentication,”  Technology 
licensed  to  BioPassword  Inc.,  Washington.  Patent  filed  through  Blakely  Sokoloff 
Taylor  &  Zafman,  LLP. 

PSU  Invention  Disclosure  No.  2003-0780. 

Technology  transferred  to  and  licensed  with  BioPassword  Inc.,  Issaquah,  WA  98027. 
The  system  is  incorporated  in  version  of  their  Enterprise  and  Network  authentication 
system. 

-  First  ever  direct  income  from  an  invention  for  Louisiana  Tech  University 

-  Appeared  in  the  national  news 

Background'.  The  multidisciplinary  research  on  anomaly  detection  and  failure  pattern 
discovery  in  Electromechanical  Systems  and  Networked  Robotic  systems,  conducted 
under  the  ARO  grant  DAAD 19-0 1-06046,  has  been  extended  to  Computer  User  Login 
Authentication.  The  objective  is  to  permit  authorized  entry  and  inhibit  unauthorized 
access  into  computer  systems  and  networks,  based  on  the  key  stroke  typing  of  individual 
users.  This  neural-network-based  authentication  algorithm  makes  a  suboptimal  trade-off 
between  the  probabilities  of  correct  authorization  and  false  through  classification  of  the 
Keystroke-typing  patterns  as  normal  or  anomalous. 

Key  Features'.  The  neural  network  architecture  in  key-stroke  pattern  classification  has 
two  types  of  connections:  (i)  weight  vector  W;  and  (ii)  dispersion  vector  V.  In  the 
learning  phase,  W  adapts  to  the  users’  keystroke  exemplars,  and  V  adapts  to  dispersion 
(e.g.,  standard  deviation)  of  the  users’  keystrokes.  The  algorithm  consists  of  three 
phases:  training,  validation,  and  testing.  The  authentication  system  leams  the  vectors  W 
and  V  in  the  training  phase.  The  system  parameters,  Scale  Factor  (SF)  and  Percentage 
Success  (PS),  are  adjusted  during  the  validation  phase.  In  the  testing  and  actual-use 
phase,  correct  classification  results  in  fine  tuning  W  and  V,  thereby  adapting  to  the  users’ 
typing  pattern.  The  weight  vector  W  and  the  dispersion  vector  V  adapt  to  the  changing 
typing  patterns  of  the  users  during  regular  use. 

Contributions'.  The  significant  contributions  of  the  neural-network-based  authentication 
algorithm  are  outlined  below: 

(1)  A  novel  neural  network  architecture  to  build  authentication  systems 

(2)  Self  adaptation  to  changes  in  typing  patterns  without  re-training  the  network 

(3)  Robustness  of  network  performance  connections  with  respect  to  user  characteristics 

(4)  Automatic  enhancement  or  relaxation  of  security  based  on  the  SF  and  PS  parameters 

(5)  Variability  of  W  and  V  keep  track  of  variability  in  the  key-stroke  patterns 
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Performance  Summary:  The  authentication  system  autonomously  learns  the  keystroke 
patterns  of  an  individual  user  and  uniquely  verifies  whether  the  user  is  a  legitimate  one  or 
an  impostor.  The  system  also  adapts  to  a  user’s  changing  typing  patterns.  A  prototype 
system,  on  the  average,  has  yielded  -1.36%  Imposter  Pass  Rate  (IPR)  and  -2.31%  False 
Authentication  Rate  (FAR).  These  results  exceed  those  of  existing  authentication 
systems. 


3.  V.  V.  Phoha,  A  Ray,  S,  Joshi,  S.  Vyuruu,  S.  Phoha,  “Hidden  Markov  Model 
("HMM")-Based  User  Authentication  Using  Keystroke  Dynamics,”  Patent  filed 
through  Blakely  Sokoloff  Taylor  &  Zafiman,  LLP. 

Background:  The  multidisciplinary  research  on  anomaly  detection  and  failure  pattern 
discovery  in  Electromechanical  Systems  and  Networked  Robotic  systems,  conducted 
under  the  ARO  grant  DAAD 19-0 1-06046,  has  been  extended  to  Computer  User  Login 
Authentication.  The  objective  is  to  permit  authorized  entry  and  inhibit  unauthorized 
access  into  computer  systems  and  networks,  based  on  the  key  stroke  typing  of  individual 
users.  This  neural-network-based  authentication  algorithm  makes  a  suboptimal  trade-off 
between  the  probabilities  of  correct  authorization  and  false  through  classification  of  the 
Keystroke-typing  patterns  as  normal  or  anomalous. 

Key  Features:  A  novel  computer  user  authentication  technique  using  Hidden  Markov 
Model  (HMM)  has  been  developed  based  on  key-stroke  pattern  classification. 
Authentication  of  a  user  is  made  in  two  stages:  (i)  user  identification  stage,  wherein  the 
user’s  identity  is  detennined  by  the  maximum  probability  score  for  the  given  keystroke 
pattern,  and  (ii)  user  verification  stage,  wherein  the  probability  score  for  the  given 
keystroke  pattern  is  examined  for  a  claimed  user.  Decisions  on  the  authenticity  of  a  user 
are  made  based  on  the  results  of  both  the  stages  and  threshold  criteria. 

Contributions:  The  significant  contributions  of  the  HMM -based  authentication  algorithm 
are  outlined  below: 

(1)  A  novel  HMM  architecture  to  build  authentication  systems 

(2)  Self  adaptation  to  changes  in  typing  patterns  without  re-training  the  network 

(3)  Robustness  of  network  performance  connections  with  respect  to  user  characteristics 

Performance  Summary:  The  authentication  system  autonomously  learns  the  keystroke 
patterns  of  an  individual  user  and  uniquely  verifies  whether  the  user  is  a  legitimate  one  or 
an  impostor.  The  system  also  adapts  to  a  user’s  changing  typing  patterns.  A  prototype 
system,  on  the  average,  has  yielded  a  false  accept  rate  of  -0.74%  and  false  reject  rate  of 
-8.06%. 


16 


APPENDICES 


Selected  papers  published  in  peer 
reviewed  journals 


17 


ELSEVIER 


Available  at 


www.ElsevierComputerScience.com 


POWERED  BY  SCIENCE 


& 


DIRECT* 


SIGNAL 

PROCESSING 


Signal  Processing  84  (2004)  1115-1130 


www.elsevier.com/locate/sigpro 


Symbolic  dynamic  analysis  of  complex  systems  for 

anomaly  detection *  *r 

Asok  Ray* 

Mechanical  Engineering  Department,  137,  Reher  Building,  The  Pennsylvania  State  University,  University  Park,  PA  16802,  USA 

Received  2  October  2003 


Abstract 

This  paper  presents  a  novel  concept  of  anomaly  detection  in  complex  dynamical  systems  using  tools  of  Symbolic  Dynamics, 
Finite  State  Automata,  and  Pattern  Recognition,  where  time-series  data  of  the  observed  variables  on  the  fast  time-scale 
are  analyzed  at  slow  time-scale  epochs  for  early  detection  of  (possible)  anomalies.  The  concept  of  anomaly  detection  in 
dynamical  systems  is  elucidated  based  on  experimental  data  that  have  been  generated  from  an  active  electronic  circuit  with 
a  slowly  varying  dissipation  parameter. 

©  2004  Elsevier  B.V.  All  rights  reserved. 

Keywords:  Fault  detection;  Symbolic  dynamics;  Pattern  recognition;  Complex  systems 


1.  Introduction 

Anomaly  in  a  dynamical  system  is  defined  as  a 
deviation  from  its  nominal  behavior  and  can  be  as¬ 
sociated  with  parametric  or  non-parametric  changes 
that  may  gradually  evolve  in  the  system.  Early  detec¬ 
tion  of  anomalies  in  complex  dynamical  systems  is 
essential  not  only  for  prevention  of  cascading  catas¬ 
trophic  failures,  but  also  for  enhancement  of  perfor¬ 
mance  and  availability  [16].  For  anomaly  detection,  it 
might  be  necessary  to  rely  on  time-series  data  gener¬ 
ated  from  sensors  and  other  sources  of  information  [  1  ] , 
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because  accurate  and  computationally  tractable  mod¬ 
elling  of  complex  system  dynamics  is  often  infea¬ 
sible  solely  based  on  the  fundamental  principles  of 
physics. 

This  paper  formulates  and  validates,  by  labora¬ 
tory  experimentation,  a  novel  concept  for  detection 
of  slowly  evolving  anomalies  in  complex  dynamical 
systems.  Often  such  dynamical  systems  are  either 
self-excited  or  can  be  stimulated  with  a  priori  known 
exogenous  inputs  to  recognize  (possible)  anomaly 
patterns  from  the  observed  stationary  response.  Early 
detection  of  an  anomaly  (i.e.,  small  parametric  or 
non-parametric  changes)  has  motivated  formulation 
and  validation  of  the  proposed  Symbolic  Dynamic 
approach  to  pattern  recognition,  which  is  based  on 
the  following  assumptions: 

•  The  system  behavior  is  stationary  at  the  fast  time 

scale  of  the  process  dynamics; 


0165-1684/$  -  see  front  matter  ©  2004  Elsevier  B.V.  All  rights  reserved, 
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•  An  observable  non- stationary  behavior  of  the  dy¬ 
namical  system  can  be  associated  with  anomaly(ies) 

evolving  at  a  slow  time  scale. 

The  theme  of  anomaly  detection,  formulated  in  this 
paper,  is  built  upon  the  concepts  of  Symbolic  Dynam¬ 
ics  [14,15]  Finite  State  Automata  [12],  and  Pattern 
Recognition  [9]  as  a  means  to  qualitatively  describe 
the  (fast-time-scale)  dynamical  behavior  in  terms 
of  symbol  sequences  [2,4],  Appropriate  phase-space 
partitioning  of  the  dynamical  system  yields  an  alpha¬ 
bet  to  obtain  symbol  sequences  from  time-series  data 
[1,8,13].  Then,  tools  of  computational  mechanics  [7] 
are  used  to  identify  statistical  patterns  in  these  sym¬ 
bolic  sequences  through  construction  of  a  (probabilis¬ 
tic)  finite-state  machine  from  each  symbol  sequence. 
Transition  probability  matrices  of  the  finite-state  ma¬ 
chines,  obtained  from  the  symbol  sequences,  capture 
the  pattern  of  the  system  behavior  by  information 
compression.  For  anomaly  detection,  it  suffices  that  a 
detectable  change  in  the  pattern  represents  a  deviation 
of  the  nominal  behavior  from  an  anomalous  one.  The 
state  probability  vectors,  which  are  derived  from  the 
respective  state  transition  matrices  under  the  nominal 
and  an  anomalous  condition,  yield  a  vector  measure  of 
the  anomaly,  which  provides  more  information  than  a 
scalar  measure  such  as  the  complexity  measure  [20]. 

In  contrast  to  the  e-machine  [7,20]  that  has  an  a 
priori  unknown  structure  and  yields  optimal  pattern 
discovery  in  the  sense  of  mutual  information  [5,11], 
the  state  machine  adopted  in  this  paper  has  an  a  priori 
known  structure  that  can  be  freely  chosen.  Although 
the  proposed  approach  is  suboptimal,  it  provides  a 
common  state  machine  structure  where  physical  sig¬ 
nificance  of  each  state  is  invariant  under  changes  in 
the  statistical  patterns  of  symbol  sequences.  This  fea¬ 
ture  allows  unambiguous  detection  of  possible  anoma¬ 
lies  from  symbol  sequences  at  different  (slow-time) 
epochs.  The  proposed  approach  is  apparently  compu¬ 
tationally  faster  than  the  e -machine  [20],  because  of 
significantly  fewer  number  of  floating  point  arithmetic 
operations.  These  are  the  motivating  factors  for  in¬ 
troducing  this  new  anomaly  detection  concept  that  is 
based  on  a  fixed- structure  fixed-order  Markov  chain, 
called  the  /J-Markov  machine  in  the  sequel. 

The  anomaly  detection  problem  is  separated  into 
two  parts  [21]:  (i)  forward  problem  of  Pattern 
Discovery  to  identify  variations  in  the  anomalous 


behavior  patterns,  compared  to  those  of  the  nominal 
behavior;  and  (ii)  inverse  problem  of  Pattern  Recog¬ 
nition  to  infer  parametric  or  non-parametric  changes 
based  on  the  learnt  patterns  and  observed  stationary 
response.  The  inverse  problem  could  be  ill-posed  or 
have  no  unique  solution.  That  is,  it  may  not  always 
be  possible  to  identify  a  unique  anomaly  pattern 
based  on  the  observed  behavior  of  the  dynamical 
system.  Nevertheless,  the  feasible  range  of  parameter 
variation  estimates  can  be  narrowed  down  from  the 
intersection  of  the  information  generated  from  inverse 
images  of  the  responses  under  several  stimuli. 

It  is  envisioned  that  complex  dynamical  systems 
will  acquire  the  ability  of  self-diagnostics  through  us¬ 
age  of  the  proposed  anomaly  detection  technique  that 
is  analogous  to  the  diagnostic  procedure  employed  in 
medical  practice  in  the  following  sense.  Similar  to 
the  notion  of  injecting  medication  or  inoculation  on  a 
nominally  healthy  patient,  a  dynamical  system  would 
be  excited  with  known  stimuli  (chosen  in  the  for¬ 
ward  problem)  in  the  idle  cycles  for  self  diagnosis  and 
health  monitoring.  The  inferred  information  on  health 
status  can  then  be  used  for  the  purpose  of  self-healing 
control  or  life-extending  control  [25].  This  paper  fo¬ 
cuses  on  the  forward  problem  and  demonstrates  the 
efficacy  of  anomaly  detection  based  on  experimental 
data  generated  from  an  active  electronic  circuit  with 
a  slowly  varying  dissipation  parameter. 

The  paper  is  organized  in  seven  sections  and  two 
appendices.  Section  2  briefly  introduces  the  notion 
of  nonlinear  time-series  analysis.  Section  3  pro¬ 
vides  a  brief  overview  of  symbolic  dynamics  and 
encoding  of  time  series  data.  Section  4  presents  two 
ensemble  approaches  for  statistical  pattern  represen¬ 
tation.  It  also  presents  information  extraction  based 
on  the  e-machine  [7]  and  the  /J-Markov  machine, 
as  well  as  their  comparison  from  different  perspec¬ 
tives.  Section  5  presents  the  notion  of  anomaly 
measure  to  quantify  the  changing  patterns  of  anoma¬ 
lous  behavior  of  the  dynamical  system  form  the 
information-theoretic  perspectives,  followed  by  an 
outline  of  the  anomaly  detection  procedure.  Sec¬ 
tion  6  presents  experimental  results  on  a  nonlinear 
active  electronic  circuit  to  demonstrate  efficacy  of 
the  proposed  anomaly  detection  technique.  Section 
7  summarizes  and  concludes  the  paper  with  recom¬ 
mendations  for  future  research.  Appendix  A  explains 
the  physical  significance  of  information-theoretic 
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quantities  used  in  the  Section  4.1  and  Section  5. 
Appendix  B  introduces  the  concept  of  shift  spaces, 
which  is  used  to  delineate  the  differences 
between  the  e-machine  [7]  and  the  /J- Markov  machine 
in  Section  4.4. 


2.  Nonlinear  time-series  analysis 

This  section  presents  nonlinear  time-series  analysis 
(NTSA)  that  is  needed  to  extract  relevant  physical  in¬ 
formation  on  the  dynamical  system  from  the  observed 
data.  NTSA  techniques  are  usually  executed  in  the 
following  steps  [1]: 

1.  Signal  separation:  The  (deterministic)  time- 
dependent  signal  {y(n)  :  n  £  Id },  where  Id  is  the  set 
of  positive  integers,  is  separated  from  noise,  using 
time-frequency  and  other  types  of  analysis. 

2.  Phase  space  reconstruction:  Based  on  the  Tak- 
ens  Embedding  theorem  [22],  time  lagged  or  delayed 
variables  are  used  to  construct  the  state  vector  x(«) 
in  a  phase  space  of  dimension  dE  (which  is  diffeo- 
morphically  equivalent  to  the  attractor  of  the  original 
dynamical  system)  as  follows: 

5L(n)  =  [y(n),y(n  +  T\...,y(n  +  (<dE-\)T)l  (1) 

where  the  time  lag  T  is  determined  using  mutual  in¬ 
formation ;  and  one  of  the  ways  to  determine  dE  is  the 
false  nearest  neighbors  test  [1]. 

3.  Signal  classification:  Signal  classification  and 
system  identification  in  nonlinear  chaotic  systems  re¬ 
quire  a  set  of  invariants  for  each  subsystem  of  interest 
followed  by  comparison  of  observations  with  those 
in  the  library  of  invariants.  The  invariants  are  proper¬ 
ties  of  the  attractor  and  could  be  independent  of  any 
particular  trajectory.  These  invariants  can  be  divided 
into  two  classes:  fractal  dimensions  and  Lyapunov 
exponents.  Fractal  dimensions  characterize  geometri¬ 
cal  complexity  of  dynamics  (e.g.,  spatial  distribution 
of  points  along  a  system  orbit);  and  Lyapunov  expo¬ 
nents  describe  the  dynamical  complexity  (e.g.,  stretch¬ 
ing  and  folding  of  an  orbit  in  the  phase  space)  [18]. 

4.  Modeling  and  prediction:  This  step  involves 
determination  of  the  parameters  of  the  assumed  model 
of  the  dynamics,  which  is  consistent  with  the  invari¬ 
ant  classifiers  (e.g.,  Lyapunov  exponents,  and  fractal 
dimensions). 


The  first  three  steps  show  how  chaotic  systems  may 
be  separated  from  stochastic  ones  and,  at  the  same 
time,  provide  estimates  of  the  degrees  of  freedom 
and  the  complexity  of  the  underlying  dynamical  sys¬ 
tem.  Based  on  this  information,  Step  4  formulates  a 
state-space  model  that  can  be  used  for  prediction  of 
anomalies  and  incipient  failures.  The  functional  form 
often  used  in  this  step,  includes  orthogonal  polynomi¬ 
als  and  radial  basis  functions.  This  paper  has  adopted 
an  alternative  class  of  discrete  models  inspired  from 
Automata  Theory,  which  is  built  upon  the  principles 
of  Symbolic  Dynamics  as  described  in  the  following 
section. 

3.  Symbolic  dynamics  and  encoding 

This  section  introduces  the  concept  of  Symbolic 
Dynamics  and  its  usage  for  encoding  nonlinear  sys¬ 
tem  dynamics  from  observed  time-series  data.  Let  a 
continuously  varying  physical  process  be  modelled  as 
a  finite-dimensional  dynamical  system  in  the  setting 
of  an  initial  value  problem: 

=/(x(t),0(4);  x(0)  =  x0,  (2) 

where  t  £  [0,  oo)  denotes  the  (fast-scale)  time;  x  £  W 
is  the  state  vector  in  the  phase  space;  and  0  £  IRK  is 
the  (possibly  anomalous)  parameter  vector  varying 
in  (slow-scale)  time  ts.  Sole  usage  of  the  model  in 
Eq.  (2)  may  not  always  be  feasible  due  to  unknown 
parametric  and  non-parametric  uncertainties  and 
noise.  A  convenient  way  of  learning  the  dynami¬ 
cal  behavior  is  to  rely  on  the  additional  information 
provided  by  (sensor-based)  time-series  data  [1,4]. 

A  tool  for  behavior  description  of  nonlinear  dynam¬ 
ical  systems  is  based  on  the  concept  of  formal  lan¬ 
guages  for  transitions  from  smooth  dynamics  to  a  dis¬ 
crete  symbolic  description  [2].  The  phase  space  of  the 
dynamical  system  in  Eq.  (2)  is  partitioned  into  a  finite 
number  of  cells,  so  as  to  obtain  a  coordinate  grid  of 
the  space.  A  compact  (i.e.,  closed  and  bounded)  re¬ 
gion  Q  £  IR",  within  which  the  (stationary)  motion  un¬ 
der  the  specific  exogenous  stimulus  is  circumscribed, 
is  identified.  Encoding  of  02  is  accomplished  by  intro¬ 
ducing  a  partition  EE  =  i}  consisting  of 

m  mutually  exclusive  (i.e.,  Bj  CiB/,  =  0  V/'  f  k),  and 
exhaustive  (i.e.,  Uy^o  =  cells.  The  dynamical 
system  describes  an  orbit  by  the  time-series  data  as: 
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O  =  {xq,x\  ■  ■  ■  ,Xk  ■  ■  • } , x,  G  Q,  which  passes  through 
or  touches  the  cells  of  the  partition  EE. 

Let  us  denote  the  cell  visited  by  the  trajectory  at  a 
time  instant  as  a  random  variable  .S  that  takes  a  sym¬ 
bol  value  s  £  .  The  set  sz/  of  m  distinct  symbols  that 

label  the  partition  elements  is  called  the  symbol  alpha¬ 
bet.  Each  initial  state  xq&£2  generates  a  sequence  of 
symbols  defined  by  a  mapping  from  the  phase  space 
into  the  symbol  space  as: 

*o  ?  'S' / o 'V / 1 Sp_  •  •  •  Sik  .  (3) 

The  mapping  in  Eq.  (3)  is  called  Symbolic  Dynam¬ 
ics  as  it  attributes  a  legal  (i.e.,  physically  admissible) 
symbol  sequence  to  the  system  dynamics  starting  from 
an  initial  state.  (Note:  A  symbol  alphabet  s/  is  called 
a  generating  partition  of  the  phase  space  £2  if  every 
legal  symbol  sequence  uniquely  determines  a  specific 
initial  condition  x0,  i.e.,  every  symbolic  orbit  uniquely 
identifies  one  continuous  space  orbit.)  Fig.  1  pictori- 
ally  elucidates  the  concepts  of  partitioning  a  finite  re¬ 
gion  of  the  phase  space  and  mapping  from  the  parti¬ 
tioned  space  into  the  symbol  alphabet.  This  represents 
a  spatial  and  temporal  discretization  of  the  system 
dynamics  defined  by  the  trajectories.  Fig.  1  also  shows 
conversion  of  the  symbol  sequence  into  a  finite-state 
machine  as  explained  in  later  sections. 

Symbolic  dynamics  can  be  viewed  as  coarse  grain¬ 
ing  of  the  phase  space,  which  is  subjected  to  (possible) 
loss  of  information  resulting  from  granular  impreci¬ 
sion  of  partitioning  boxes,  measurement  noise  and 
errors,  and  sensitivity  to  initial  conditions.  Flowever, 
the  essential  robust  features  (e.g.,  periodicity  and 
chaotic  behavior  of  an  orbit)  are  expected  to  be  pre¬ 
served  in  the  symbol  sequences  through  an  appropri¬ 
ate  partitioning  of  the  phase  space  [2].  Although  the 
theory  of  phase-space  partitioning  is  well  developed 


for  one-dimensional  mappings,  very  few  results  are 
known  for  two-  and  higher-dimensional  systems  [4]. 

4.  Pattern  identification 

Given  the  intricacy  of  phase  trajectories  in  complex 
dynamical  systems,  the  challenge  is  to  identify  their 
patterns  in  an  appropriate  category  by  using  one  of 
the  following  two  alternative  approaches: 

•  The  single-item  approach,  which  relies  on  Kol¬ 
mogorov  Chiatin  (KC)  complexity,  also  known  as 
algorithmic  complexity  [5],  for  exact  pattern  regen¬ 
eration; 

•  The  ensemble  approach,  which  regards  the  pattern 
as  one  of  many  possible  experimental  outcomes, 
for  estimated  pattern  regeneration. 

While  the  single-item  approach  is  common  in  coding 
theory  and  computer  science,  the  ensemble  approach 
has  been  adopted  in  this  paper  due  to  its  physical 
and  statistical  relevance.  As  some  of  the  legal  sym¬ 
bol  sequences  may  occur  more  frequently  than  oth¬ 
ers,  a  probability  is  attributed  to  each  observed  se¬ 
quence.  The  collection  of  all  legal  symbol  sequences 
S—m  ■  ■  ■  S-2S~iSoSi  ■  ■  ■  Sn,  N,M  =  0, 1,2 . . .,  defines 
a  stochastic  process  that  is  a  symbolic  probabilistic 
description  of  the  continuous  system  dynamics. 

Let  us  symbolically  denote  a  discrete-time, 
discrete-valued  stochastic  process  as 

S  =  ---,S.2S-MS2---,  (4) 

where  each  random  variable  .S',  takes  exactly  one 
value  in  the  (finite)  alphabet  si  of  m  symbols  (see 
Section  3).  The  symbolic  stochastic  process  §  is  de¬ 
pendent  on  the  specific  partitioning  of  the  phase  space 
and  is  non-Markovian,  in  general.  Even  if  a  partition¬ 
ing  that  makes  the  stochastic  process  a  Markov  chain 
exists,  identification  of  such  a  partitioning  is  not  al¬ 
ways  feasible  because  the  individual  cells  may  have 
fractal  boundaries  instead  of  being  simple  geometri¬ 
cal  objects.  In  essence,  there  is  a  trade-off  between 
selecting  a  simple  partitioning  leading  to  a  compli¬ 
cated  stochastic  process,  and  a  complicated  partition¬ 
ing  leading  to  a  simple  stochastic  process.  Recent 
literature  has  reported  a  comprehensive  numerical  pro¬ 
cedure  for  construction  phase-space  partitions  from 
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the  time-series  data  [13].  Having  defined  a  partition 
of  the  phase  space,  the  time-series  data  is  converted 
to  a  symbol  sequence  that,  in  turn,  is  used  for  con¬ 
struction  of  a  finite-state  machine  using  the  tools  of 
Computational  Mechanics  [7]  as  illustrated  in  Fig.  1. 

This  paper  considers  two  alternative  techniques  of 
finite-state  machine  construction  from  a  given  sym¬ 
bol  sequence  EC:  (i)  the  £-machine  formulation  [20]; 
and  (ii)  a  new  concept  based  on  £>th  order  Markov 
chains,  called  the  D-Markov  machine,  for  identifying 
patterns  based  on  time  series  analysis  of  the  observed 
data.  Both  techniques  rely  on  information-theoretic 
principles  (see  Appendix  A)  and  are  based  on 
computational  mechanics  [7]. 

4.1.  The  e-machine 

Like  statistical  mechanics  [10,4],  computational 
mechanics  is  concerned  with  dynamical  systems 
consisting  of  many  partially  correlated  components. 
Whereas  Statistical  Mechanics  deals  with  the  local 
space-time  behavior  and  interactions  of  the  system 
elements,  computational  mechanics  relies  on  the  joint 
probability  distribution  of  the  phase-space  trajectories 
of  a  dynamical  system.  The  e-machine  construction 
[7,20]  makes  use  of  the  joint  probability  distribution 
to  infer  the  information  processing  being  performed 
by  the  dynamical  system.  This  is  developed  using 
the  statistical  mechanics  of  orbit  ensembles,  rather 
than  focusing  on  the  computational  complexity  of 
individual  orbits. 

Let  the  symbolic  representation  of  a  discrete-time, 
discrete-valued  stochastic  process  be  denoted  by:  §  = 

■  •  •  S-2S-\SqS\S2  -  as  defined  earlier  in  Section  4. 
At  any  instant  t,  this  sequence  of  random  variables  can 

be  split  into  a  sequence  Si  of  the  past  and  a  sequence 
S I  of  the  future.  Assuming  conditional  stationarity  of 
the  symbolic  process  S  (i.e.,  -Parish  =  5]  being  in¬ 
dependent  of  t ),  the  subscript  t  can  be  dropped  to  de¬ 
note  the  past  and  future  sequences  as  S  and  S ,  respec¬ 
tively.  A  symbol  string,  made  of  the  first  L  symbols 

of  S,  is  denoted  by  S1'-  Similarly,  a  symbol  string, 
made  of  the  last  L  symbols  of  S,  is  denoted  by  SL- 

Prediction  of  the  future  S  necessitates  determina¬ 
tion  of  its  probability  conditioned  on  the  past  S ,  which 
requires  existence  of  a  function  e  mapping  histories 


5  to  predictions  P(S  \  s  ).  In  essence,  a  prediction  im¬ 
poses  a  partition  on  the  set  S  of  all  histories.  The  cells 
of  this  partition  contain  histories  for  which  the  same 
prediction  is  made  and  are  called  the  effective  states 
of  the  process  under  the  given  predictor.  The  set  of 
elfective  states  is  denoted  by  R;  a  random  variable  for 
an  effective  state  is  denoted  by  0t  and  its  realization 
by  p. 

The  objective  of  e-machine  construction  is  to  find 
a  predictor  that  is  an  optimal  partition  of  the  set  S  of 
histories,  which  requires  invoking  two  criteria  in  the 
theory  of  Computational  Mechanics  [6] : 

1.  Optimal  Prediction:  For  any  partition  of  his¬ 
tories  or  effective  states  01,  the  conditional  entropy 

H[SL\m>H[SL\S],  VL&N,  VS£  S,  is  equiva¬ 
lent  to  remembering  the  whole  past.  Effective  states  (0 
are  called  prescient  if  the  equality  is  attained  Vi  €  N. 
Therefore,  optimal  prediction  needs  the  effective  states 
to  be  prescient. 

2.  Principle  of  Occam  Razor :  The  prescient  states 
with  the  least  complexity  are  selected,  where  complex¬ 
ity  is  defined  as  the  measured  Shannon  information  of 
the  effective  states: 

H[0]  =  -J2P(!M  =  p)logP{0l  =  p).  (5) 

pGR 

Eq.  (5)  measures  the  amount  of  past  information 
needed  for  future  prediction  and  is  known  as  Statisti¬ 
cal  Complexity  denoted  by  C;,(7L)  (see  Appendix  A). 

For  each  symbolic  process  S,  there  is  a  unique  set  of 
prescient  states  known  as  causal  states  that  minimize 
the  statistical  complexity  Cj,(0t). 

Definition  4.1  (Shalizi  et  al.  [20]).  Let  §  be  a  (con¬ 
ditionally)  stationary  symbolic  process  and  S  be  the 
set  of  histories.  Let  a  mapping  e:  S  — >  T(S)  from 
the  set  S  of  histories  into  a  collection  T(  S)  of  mea¬ 
surable  subsets  of  S  be  defined  as: 

vre  7(s),  £(j)g{j'£S  such  that 

P(S  er\s  =  7)=P(S  er\s  =  V)}.  (6) 

Then,  the  members  of  the  range  of  the  function  e  are 
called  the  causal  states  of  the  symbolic  process  §. 
The  z'th  causal  state  is  denoted  by  q,  and  the  set  of 
all  causal  states  by  Q  C  T(  S  )•  The  random  variable 
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corresponding  to  a  causal  state  is  denoted  by  2L  and 
its  realization  by  q. 

Given  an  initial  causal  state  and  the  next  symbol 
from  the  symbolic  process,  only  successor  causal 
states  are  possible.  This  is  represented  by  the  legal 
transitions  among  the  causal  states,  and  the  probabil¬ 
ities  of  these  transitions.  Specifically,  the  probability 
of  transition  from  state  q,  to  state  qj  on  a  single 
symbol  s  is  expressed  as: 

Cf  =  PCs'  =s,  2!  =qj\£  =  q,)  Mqh  q,  £  Q,  (7) 

EE4)  =  1-  (8) 

qj  £  Q 

The  combination  of  causal  states  and  transitions  is 
called  the  e- machine  (also  known  as  the  causal  state 
model  [20])  of  a  given  symbolic  process.  Thus,  the 
s-machine  represents  the  way  in  which  the  symbolic 
process  stores  and  transforms  information.  It  also  pro¬ 
vides  a  description  of  the  pattern  or  regularities  in  the 
process,  in  the  sense  that  the  pattern  is  an  algebraic 
structure  determined  by  the  causal  states  and  their  tran¬ 
sitions.  The  set  of  labelled  transition  probabilities  can 
be  used  to  obtain  a  stochastic  matrix  [3]  given  by: 
S~=Y^se ,,/  S~s  where  the  square  matrix  ?TS  is  defined 
as:  ,TS  =  [T-j]  Ms  £  ,r// .  Denoting  p  as  the  left  eigen¬ 
vector  of  S',  corresponding  to  the  eigenvalue  X  =  1, 
the  probability  of  being  in  a  particular  causal  state  can 
be  obtained  by  nonnalizing  ||p||^,  =  1.  A  procedure 
for  construction  of  the  e-machine  is  outlined  below. 

The  original  e-machine  construction  algorithm  is  the 
subtree-merging  algorithm  as  introduced  in  [7,6],  The 
default  assumption  of  this  technique  was  employed 
by  Surana  et  al.  [21]  for  anomaly  detection.  This  ap¬ 
proach  has  several  shortcomings,  such  as  lack  of  a  sys¬ 
tematic  procedure  for  choosing  the  algorithm  param¬ 
eters,  may  return  non-deterministic  causal  states,  and 
also  suffers  from  slow  convergence  rates.  Recently, 
Shalizi  et  al.  [20]  have  developed  a  code  known  as 
Causal  State  Splitting  Reconstruction  (CSSR)  that  is 
based  on  state  splitting  instead  of  state  merging  as  was 
done  in  the  earlier  algorithm  of  subtree-merging  [7], 
The  CSSR  algorithm  starts  with  a  simple  model  for 
the  symbolic  process  and  elaborates  the  model  com¬ 
ponents  only  when  statistically  justified.  Initially,  the 
algorithm  assumes  the  process  to  be  independent  and 
identically  distributed  (iid)  that  can  be  represented  by 


a  single  causal  state  and  hence  zero  statistical  com¬ 
plexity  and  high  entropy  rate.  At  this  stage,  CSSR  uses 
statistical  tests  to  determine  when  it  must  add  states 
to  the  model,  which  increases  the  estimated  complex¬ 
ity,  while  lowering  the  entropy  rate  hfl  (see  Appendix 
A).  A  key  and  distinguishing  feature  of  the  CSSR 
code  is  that  it  maintains  homogeneity  of  the  causal 
states  and  deterministic  state-to-state  transitions  as  the 
model  grows.  Complexity  of  the  CSSR  algorithm  is: 
0(mim“)  +  0(/w2z'max+1 )  +  0(A),  where  m  is  the  size 
of  the  alphabet  ,a/;  N  is  the  data  size  and  Tmax  is  the 
length  of  the  longest  history  to  be  considered.  Details 
are  given  in  [20], 

4.2.  The  suboptimal  D-markov  machine 

This  section  presents  a  new  alternative  approach  for 
representing  the  pattern  in  a  symbolic  process,  which 
is  motivated  from  the  perspective  of  anomaly  detec¬ 
tion.  The  core  assumption  here  is  that  the  symbolic 
process  can  be  represented  to  a  desired  level  of  accu¬ 
racy  as  a  /Jth  order  Markov  chain,  by  appropriately 
choosing  deN. 

Definition  4.2.  A  stochastic  symbolic  stationary  pro¬ 
cess  §  =  •  •  ■  S-2S- 1  .S’o.S’i  S2  •  •  •  is  called  /Jth  order 
Markov  process  if  the  probability  of  the  next  symbol 
depends  only  on  the  previous  (at  most)  D  symbols, 
i.e.  the  following  condition  holds: 

m|S,-iS,-2  •  •  •  S,-_D  ■  •  •)  =  />(£!$_!  •  •  ■  S,_z>)  (9) 

Alternatively,  symbol  strings  S ,  S'  £  S  become  indis¬ 
tinguishable  whenever  the  respective  substrings  SD 

and  S,D ,  made  of  the  most  recent  D  symbols,  are  iden¬ 
tical. 

Definition  (4.2)  can  be  interpreted  as  follows: 

V  S,  S' £  S  such  that  \S\^D  and  |5"|  ^  D, 
{S'  £e{S)  and  S  £  £(/>'))  iff  S D  =  S'D.  Thus,  a  set 
{  SL  ■  L  D}  of  symbol  stings  can  be  partitioned  into 
a  maximum  of  \stf\D  equivalence  classes  where  ■<//  is 
the  symbol  alphabet,  under  the  equivalence  relation 
defined  in  Eq.  (6).  Each  symbol  string  in  {  SL'.  L  ^  D} 
either  belongs  to  one  of  the  \sd\D  equivalence  classes 
or  has  a  distinct  equivalence  class.  All  such  symbol 
strings  belonging  to  the  distinct  equivalence  class 
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form  transient  states,  and  would  not  be  of  concern  to 
anomaly  detection  for  a  (fast-time- scale)  stationary 
condition  under  (slowly  changing)  anomalies.  Given 
dgN  and  a  symbol  string  s  with  \s\=D,  the  effec¬ 
tive  state  q(D,  s  )  is  the  equivalence  class  of  symbol 
strings  as  defined  below: 

#,s)  =  {S6S:  SD  =  *s}  (10) 

and  the  set  Q(//)  of  effective  states  of  the  symbolic 
process  is  the  collection  of  all  such  equivalence 
classes.  That  is, 

Q(Z>)  =  {9(A*):  *  eS"}  (11) 

and  hence  |Q(£))|  =  si  \ D .  A  random  variable  for  a 
state  in  the  above  set  Q  of  states  is  denoted  by  2L  and 
the  /th  state  as  qj.  The  probability  of  transitions  from 
state  qj  to  state  qk  is  defined  as: 

njk  =  P(s  £  S 1  |  qj  £  Q,  (s,qj)-+qk); 

5>a  =  1;  (12) 

k 

Given  an  initial  state  and  the  next  symbol  from  the 
original  process,  only  certain  successor  states  are  ac¬ 
cessible.  This  is  represented  as  the  allowed  state  tran¬ 
sitions  resulting  from  a  single  symbol.  Note  that  n,j=0 
if  S2S3  ■  ■  ■  sq  f  s\  ■  ■  ■  s'D_  j  whenever  qt  =  s\S2  ■  ■  ■  so 
and  qj  =  s[s'2  ■  ■  ■  s'D.  Thus,  for  a  D-Markov  machine, 
the  stochastic  matrix  77  =  |  n,j  |  becomes  a  branded 
matrix  with  at  most  .r/|w+i  nonzero  entries. 

The  construction  of  a  /J- Markov  machine  is  fairly 
straightforward.  Given/)  £  hi,  the  states  are  as  defined 
in  Eqs.  (10)  and  (11).  On  a  given  symbol  sequence 
y ,  a  window  of  length  (D  +  1 )  is  slided  by  keeping 
a  count  of  occurrences  of  sequences  .v,  •  •  •  stllsln^ 
and  •  •  •  s,n  which  are,  respectively,  denoted  by 
N(sit  ■  ■  ■  sinsin+l )  and  N(sit  ■  ■  ■  sio).  Note  that  if 
N(sh  ■  ■  -SjD)  =  0,  then  the  state  q  =  sit  ■  ■  ■  sio  £  Q  has 
zero  probability  of  occurrence.  For  N(sii  ■  ■  ■  sIn )  /  0 ), 
the  transitions  probabilities  are  then  obtained  by  these 
frequency  counts  as  follows: 

_  P(sh  ■  ■  ■  siDs )  _  N(su  ■  ■  ■  siDs) 

Jk  P(sh  sio )  ~  N(sh  ■■■siD)’ 

where  the  corresponding  states  are  denoted  by:  qj  = 
shsh  •  •  •  siD  and  qk  =  sh  ■  ■  ■  sios. 

As  an  example,  Fig.  2  shows  the  finite-state  ma¬ 
chine  and  the  associated  state  transition  matrix  for  a 
//-Markov  machine,  where  the  alphabet  =  {(),!}, 


i.e.,  alphabet  size  ..s/  =  2;  and  the  states  are  chosen 
as  words  of  length  D  =  2  from  a  symbol  sequence  (/J . 
Consequently,  the  total  number  of  states  is  |  sJ\D  =  4, 
which  is  the  number  of  permutations  of  the  alphabet 
symbols  within  a  word  of  length  D\  and  the  set  of 
states  Q  =  (00,01, 10, 11}.  The  state  transition  ma¬ 
trix  on  the  right  half  of  Fig.  2  denotes  the  probability 
it q  =  pij  of  occurrence  of  the  symbol  0  £  s4  at  the 
state  q  =  ij,  where  i,j  £  si .  The  states  are  joined  by 
edges  labelled  by  a  symbol  in  the  alphabet.  The  state 
machine  moves  from  one  state  to  another  upon  oc¬ 
currence  of  an  event  as  a  new  symbol  in  the  symbol 
sequence  is  received  and  the  resulting  transition  ma¬ 
trix  has  at  most  | si\D+x  =  8  non-zero  entries.  The  ma¬ 
chine  language  is  complete  in  the  sense  that  there  are 
different  outgoing  edges  marked  by  different  symbols; 
however,  it  is  possible  that  some  of  these  arcs  may 
have  zero  probability. 


4.3.  Statistical  mechanical  concept  of  D-Markov 
machine 

This  section  outlines  an  analogy  between  the  struc¬ 
tural  features  of  the  //-Markov  machine  and  those  of 
spin  models  in  Statistical  Mechanics.  The  main  idea 
is  derived  from  the  doctoral  dissertation  of  Feldman 
[10]  who  has  demonstrated  how  measures  of  patterns 
from  Information  Theory  and  Computational  Mechan¬ 
ics  are  captured  in  the  construction  of  e-Machines.  In 
general,  the  effects  of  an  anomaly  are  reflected  in  the 
respective  state  transition  matrices.  Thus,  the  structure 
of  the  finite-state  machine  is  fixed  for  a  given  alpha¬ 
bet  size  \s/\  and  window  length  D.  Furthermore,  the 
number  of  edges  is  also  finite  because  of  the  finite  al¬ 
phabet  size.  The  elements  of  the  state  transition  matrix 
(that  is  a  stochastic  matrix  [3])  are  identified  from  the 
symbol  sequence. 
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For  \s£\  =  2  and  D  =  2,  the  finite-state  machine 
construction  is  (to  some  extent)  analogous  to  the 
one-dimensional  Ising  model  of  spin- 1/2  systems  with 
nearest  neighbor  interactions,  where  the  z-component 
of  each  spin  takes  on  one  of  the  two  possible  values 
s  =  +1  or  s  =  —  1  [10,19].  For  \s/\  ^  3,  the  machine 
would  be  analogous  to  one-dimensional  Potts  model, 
where  each  spin  is  directed  in  the  z-direction  with 
\sa/\  different  discrete  values  ,s>:  k€  1,2,  for 

a  //2-spin  model,  the  alphabet  size  \sf\  =  j  +  1  [4]. 
For  D  ^  2,  the  spin  interactions  extend  up  to  the 
(D  —  1  )th  neighbor. 

4.4.  Comparison  of  D-Markov  machine  and 
E-machine 

An  E-machine  seeks  to  find  the  patterns  in  the 
time  series  data  in  the  form  of  a  finite-state  machine, 
whose  states  are  chosen  for  optimal  prediction  of  the 
symbolic  process;  and  a  finite-state  automation  can 
be  used  as  a  pattern  for  prediction  [20].  An  alterna¬ 
tive  notion  of  the  pattern  is  one  which  can  be  used 
to  compress  the  given  observation.  The  first  notion 
of  the  pattern  subsumes  the  second,  because  the  ca¬ 
pability  of  optimal  prediction  necessarily  leads  to  the 
compression  as  seen  in  the  construction  of  states  by 
lumping  histories  together.  However,  the  converse  is 
not  true  in  general.  For  the  purpose  of  anomaly  detec¬ 
tion,  the  second  notion  of  pattern  is  sufficient  because 
the  goal  is  to  represent  and  detect  the  deviation  of 
an  anomalous  behavior  from  the  nominal  behavior. 
This  has  been  the  motivating  factor  for  proposing 
an  alternative  technique,  based  on  the  fixed  structure 
D-Markov  machine.  It  is  possible  to  detect  the  evolv¬ 
ing  anomaly,  if  any,  as  a  change  in  the  probability 
distribution  over  the  states. 

Another  distinction  between  the  D-Markov  ma¬ 
chine  and  £-machine  can  be  seen  in  terms  of finite-type 
shifts  and  sofic  shifts  [15]  (see  Appendix  B).  Basic 
distinction  between  finite-type  shifts  and  sofic  shifts 
can  be  characterized  in  terms  of  the  memory,  while  a 
finite-type  shift  has  fmiXe-length  memory,  a  sofic  shift 
uses  finite  amount  of  memory  in  representing  the 
patterns.  Flence,  finite-type  shifts  are  strictly  proper 
subsets  of  sofic  shifts.  While,  any  finite-type  shift  has 
a  representation  as  a  graph,  sofic  shifts  can  be  repre¬ 
sented  as  a  labelled  graph.  As  a  result,  the  finite-type 
shift  can  be  considered  as  an  “extreme  version”  of  a 


D-Markov  chain  (for  an  appropriate  D)  and  sofic  shifts 
as  an  “extreme  version”  of  a  Hidden  Markov  process 
[24],  respectively.  The  shifts  have  been  referred  to  as 
“extreme”  in  the  sense  that  they  specify  only  a  set  of 
allowed  sequences  of  symbols  (i.e.,  symbol  sequences 
that  are  actually  possible,  but  not  the  probabilities  of 
these  sequences).  Note  that  a  Hidden  Markov  model 
consists  of  an  internal  D-order  Markov  process  that 
is  observed  only  by  a  function  of  its  internal-state 
sequence.  This  is  analogous  to  sofic  shifts  that  are  ob¬ 
tained  by  a  labelling  function  on  the  edge  of  a  graph, 
which  otherwise  denotes  a  finite-type  shift.  Thus,  in 
these  terms,  an  £-machine  infers  the  Hidden  Markov 
Model  (sofic  shift)  for  the  observed  process.  In 
contrast,  the  D-Markov  Model  proposed  in  this  paper 
infers  a  (finite-type  shift)  approximation  of  the  (sofic 
shift)  £-machine. 


5.  Anomaly  measure  and  detection 

The  machines  described  in  Sections  4. 1  and  4.2  rec¬ 
ognize  patterns  in  the  behavior  of  a  dynamical  system 
that  undergoes  anomalous  behavior.  In  order  to  quan¬ 
tify  changes  in  the  patterns  that  are  representations  of 
evolving  anomalies,  we  induce  an  anomaly  measure 
on  these  machines,  denoted  by  Jt.  The  anomaly  mea¬ 
sure  J'/  can  be  constructed  based  on  the  following 
information-theoretic  quantities:  entropy  rate,  excess 
entropy,  and  complexity  measure  of  a  symbol  string 
!:f  (see  Appendix  A). 

•  The  entropy  rate  /;;,(///)  quantifies  the  intrinsic  ran¬ 
domness  in  the  observed  dynamical  process. 

•  The  excess  entropy  E(.(/')  quantifies  the  memory  in 
the  observed  process. 

•  The  statistical  complexity  Vfff)  of  the  state  ma¬ 
chine  captures  the  average  memory  requirements 
for  modelling  the  complex  behavior  of  a  process. 

Given  two  symbol  strings  Sf  and  4/\u  it  is  possible  to 
obtain  a  measure  of  anomaly  by  adopting  any  one  of 
the  following  three  alternatives: 

(  or 

o))  =  j  |E(50  -  E(y’o)!,  or 
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Note  that  each  of  the  anomaly  measures,  defined 
above,  is  a  pseudo  metric  [17].  For  example,  let  us 
consider  two  periodic  processes  with  unequal  periods, 
represented  by  ff  and  //q  .  For  both  processes,  hjL  =  0, 
so  that  Jl ( //',  /To )  =  0  for  the  first  of  the  above  three 
options,  even  if  ^  £^o- 

The  above  measures  are  obtained  through  scalar¬ 
valued  functions  defined  on  a  state  machine  and  do 
not  exploit  the  rich  algebraic  structure  represented  in 
the  state  machine.  For  example,  the  comiection  matrix 
2T  associated  with  the  £-machine  (see  Section  4.1), 
can  be  treated  as  a  vector  representation  of  any  possi¬ 
ble  anomalies  in  the  dynamical  system.  The  induced 
2-norm  of  the  difference  between  the  ^-matrices  for 
the  two  state  machines  can  then  be  used  as  a  measure 
of  anomaly,  i.e.,  o)  =  || ST  —  2Tq\i-  Such  a 

measure,  used  in  [21],  was  found  to  be  effective.  How- 
ever,  there  is  some  subtlety  in  using  this  measure  on 
s-machines,  because  e-machines  do  not  guarantee  that 
the  machines  fonnulated  from  the  symbol  sequences 
//  and  Sf o  have  the  same  number  of  states;  and  these 
states  do  not  necessarily  have  similar  physical  sig¬ 
nificance.  In  general,  ST  and  STq  may  have  different 
dimensions  and  different  physical  significance.  How- 
ever,  by  encoding  the  causal  states,  ,T  could  be  em¬ 
bedded  in  a  larger  matrix,  and  an  induced  norm  of  the 
difference  between  ST  matrices  for  these  two  machines 
can  be  defined.  Alternatively,  a  (vector)  measure  of 
anomaly  can  be  derived  directly  from  the  stochastic 
matrix  ST  as  the  left  eigenvector  p  corresponding  to 
the  unit  eigenvalue  of  2T ,  which  is  the  state  probabil¬ 
ity  vector  under  a  stationary  condition. 

This  paper  has  adopted  the  ZJ-Markov  machine  ap¬ 
proach,  described  in  the  Section  4.2  to  build  the  state 
machines.  Since  /J-Markov  machines  have  a  fixed 
state  structure,  the  state  probability  vector  p  associated 
with  the  state  machine  have  been  used  for  a  vector 
representation  of  anomalies,  leading  to  the  anomaly 
measure  as  a  distance  function  between 

the  respective  probability  vectors  p  and  po  (that  are 
of  identical  dimensions),  or  any  other  appropriate 
functional. 

5.1.  Anomaly  detection  procedure 

Flaving  discussed  various  tools  and  techniques,  this 
section  outlines  the  steps  of  the  forward  problem  and 
the  inverse  problem  described  in  Section  1.  Following 


are  the  steps  for  the  forward  problem: 

(FI)  Selection  of  an  appropriate  set  of  input  stimuli. 

(F2)  Signal-noise  separation,  time  interval  selection, 
and  phase-space  construction. 

(F3)  Choice  of  a  phase  space  partitioning  to  generate 
symbol  alphabet  and  symbol  sequences. 

(F4)  State  Machine  construction  using  generated  sym¬ 
bol  sequence(s)  and  determining  the  connection 
matrix. 

(F5)  Selection  of  an  appropriate  metric  for  the 
anomaly  measure  Ji . 

(F6)  Formulation  and  calibration  of  a  (possibly 
non-parametric )  relation  between  the  computed 
anomaly  measure  and  known  physical  anomaly 
under  which  the  time-series  data  were  collected 
at  different  (slow-time)  epochs. 

Following  are  the  steps  for  the  inverse  problem: 

(11)  Excitation  with  known  input  stimuli  selected  in 
the  forward  problem. 

(12)  Generation  of  the  stationary  behavior  as 
time-series  data  for  each  input  stimulus  at  dif¬ 
ferent  (slow-time)  epochs. 

(13)  Embedding  the  time-series  data  in  the  phase 
space  determined  for  the  corresponding  input 
stimuli  in  Step  F2  of  the  forward  problem. 

(14)  Generation  of  the  symbol  sequence  using  the 
same  phase-space  partition  as  in  Step  F3  of  the 
forward  problem. 

(15 )  State  Machine  construction  using  the  symbol  se¬ 
quence  and  determining  the  anomaly  measure. 

(16)  Detection  and  identification  of  an  anomaly,  if 
any,  based  on  the  computed  anomaly  measure 
and  the  relation  derived  in  Step  F6  of  the  forward 
problem. 


6.  Application  to  an  active  electronic  circuit 

This  section  illustrates  an  application  of  the 
/J-Markov  machine  concept  for  anomaly  detection 
on  an  experimental  apparatus  that  consists  of  an  ac¬ 
tive  electronic  circuit.  The  apparatus  implements  a 
second-order  non-autonomous,  forced  Duffing  equa¬ 
tion  in  real  time  [23].  The  governing  equation  with 
a  cubic  nonlinearity  in  one  of  the  state  variables  is 
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Fig.  3.  Phase  plots  for  electronic  circuit  experiment. 


given  below: 

+  P(ts)  +x(t)  +x3(/)  =  A  cos  cot.  (14) 

The  dissipation  parameter  /i(4),  realized  in  the  form 
of  a  resistance  in  the  circuit,  is  made  to  vary  in  the 
slow  time  scale  ts  and  is  treated  as  a  constant  in  the  fast 
time  scale  t  at  which  the  dynamical  system  is  excited. 
The  goal  is  to  detect,  at  an  early  stage,  changes  in  P(ts) 
that  are  associated  with  the  anomaly. 

In  the  forward  problem,  the  first  task  is  the  selection 
of  appropriate  input  stimuli.  For  illustration  purposes, 
we  have  used  the  stimulus  with  amplitude  A  =  22.0 
and  frequency  co  =  5.0  in  this  paper.  Changes  in  the 
stationary  behavior  of  the  electronic  circuit  take  place 
starting  from  /i  «  0.10  with  significant  changes  oc¬ 
curring  in  the  narrow  range  of  0.28  <  P  <  0.29.  The 
stationary  behavior  of  the  system  response  for  this  in¬ 
put  stimulus  is  obtained  for  several  values  of  P  in  the 
range  of  0.10-0.35. 

The  four  plates  in  Fig.  3  exhibit  four  phase  plots 
for  the  values  of  the  parameter  j)  at  0.10,  0.27,  0.28, 
and  0.29,  respectively,  relating  the  phase  variable  of 
electrical  charge  that  is  proportional  to  the  voltage 
across  one  of  the  capacitors  in  the  electronic  circuit, 


Time 


Fig.  4.  Time  plots  for  electronic  circuit  experiment. 

and  its  time  derivative  (i.e.,  the  instantaneous  cur¬ 
rent).  While  a  small  difference  between  the  plots  for 
P  =  0.10  and  0.27  is  observed,  there  is  no  clearly  visi¬ 
ble  difference  between  the  plots  for  P  =  0.27  and  0.28 
in  Fig.  3.  However,  the  phase  plots  for  p  =  0.28  and 
0.29  display  a  very  large  difference  indicating  period 
doubling  possibly  due  to  onset  of  bifurcation.  Fig.  4 
displays  time  responses  of  the  stationary  behavior  of 
the  phase  variable  for  different  values  of  the  parameter 
P  corresponding  to  the  phase  plots  in  Fig.  3.  The  plots 
in  Fig.  4  are  phase-aligned  for  better  visibility.  (Note 
that  the  proposed  anomaly  detection  method  does  not 
require  phase  alignment;  equivalently,  the  finite-state 
machine  Fig.  2  can  be  started  from  any  arbitrary 
state  corresponding  to  no  specific  initial  condition.) 
While  the  time  responses  for  p  =  0.27  and  0.28  are 
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indistinguishable,  there  is  a  small  difference  between 
those  for  (3  =  0.27  and  0.10.  Similar  to  the  phase  plots 
in  Fig.  3,  the  time  responses  for  [3  =  0.28  and  0.29 
display  existence  of  period  doubling  due  to  a  possible 
bifurcation. 

Additional  exogenous  stimuli  have  been  identified, 
which  also  lead  to  significant  changes  in  the  station¬ 
ary  behavior  of  the  electronic  system  dynamics  for 
other  ranges  of  f3.  For  example,  with  the  same  ampli¬ 
tude  A  =  22.0,  stimuli  at  the  excitation  frequencies  of 
co  =  2.0  and  co  =  1.67  (not  shown  in  Figs.  3  and  4) 
detect  small  changes  in  the  ranges  of  0.18  <  (1  <  0.20 
and  0.11  <  [3  <  0.12,  respectively  [21].  These  ob¬ 
servations  reveal  that  exogenous  stimuli  at  different 
excitation  frequencies  can  be  effectively  used  for 
detection  of  small  changes  in  j3  over  an  operating 
range  (e.g.,  0.10  <  f3  <  0.35). 

Having  obtained  the  phase  plots  from  the  time-series 
data,  the  next  step  is  to  find  a  partition  of  the  phase 
space  for  symbol  sequence  generation.  This  is  a 
difficult  task  especially  if  the  time-series  data  is 
noise-contaminated.  Several  methods  of  phase-space 
partitioning  have  been  suggested  in  literature  (for 
example,  [1,8,13]).  Apparently,  there  exist  no 
well-established  procedure  for  phase-space  partition¬ 
ing  of  complex  dynamical  systems;  this  is  a  subject 
of  active  research.  In  this  paper,  we  have  introduced 
a  new  concept  of  symbol  sequence  generation,  which 
uses  wavelet  transform  to  convert  the  time-series 
data  to  time-frequency  data  for  generating  the  symbol 
sequence.  The  graphs  of  wavelet  coefficients  ver¬ 
sus  scale  at  selected  time  shifts  are  stacked  starting 
with  the  smallest  value  of  scale  and  ending  with  its 
largest  value  and  then  back  from  the  largest  value 
to  the  smallest  value  of  the  scale  at  the  next  in¬ 
stant  of  time  shift.  The  resulting  scale  series  data 
in  the  wavelet  space  is  analogous  to  the  time-series 
data  in  the  phase  space.  Then,  the  wavelet  space 
is  partitioned  into  segments  of  coefficients  on  the 
ordinate  separated  by  horizontal  lines.  The  number 
of  segments  in  a  partition  is  equal  to  the  size  of 
the  alphabet  and  each  partition  is  associated  with  a 
symbol  in  the  alphabet.  For  a  given  stimulus,  parti¬ 
tioning  of  the  wavelet  space  must  remain  invariant 
at  all  epochs  of  the  slow  time  scale.  Nevertheless, 
for  different  stimuli,  the  partitioning  could  be  chosen 
differently.  (The  concept  of  proposed  wavelet-space 
partitioning  would  require  significant  theoretical 


research  before  its  acceptance  for  application  to  a 
general  class  of  dynamical  systems  for  anomaly  de¬ 
tection;  and  its  efficacy  needs  to  be  compared  with 
that  of  existing  phase-space  partitioning  methods  such 
as  false  nearest  neighbor  partitioning  [13].) 

The  procedure,  described  in  the  subsection  IV-B 
constructs  a  D- Markov  machine  and  obtains  the  con¬ 
nection  matrix  ST  and  the  state  vector  p  from  the  sym¬ 
bol  sequence  corresponding  to  each  [3.  For  this  anal¬ 
ysis,  the  wave  space  generated  from  each  data  set 
has  been  partitioned  into  eight  (8)  segments,  which 
makes  the  alphabet  size  \s/\  =  8  to  generate  symbol  se¬ 
quences  from  the  scale  series  data.  At  each  value  of  /»', 
the  generated  symbol  sequence  has  been  used  to  con¬ 
struct  several  /J- Markov  Machines  starting  with  D=  1 
and  higher  integers.  It  is  observed  that,  the  dominant 
probabilities  of  the  state  vector  (albeit  having  differ¬ 
ent  dimensions)  for  different  values  of  D  are  virtually 
similar.  Therefore,  a  fixed-structure  /J-Markov  Ma¬ 
chine  with  alphabet  size  \s/\  =  8  and  depth  D  =  1, 
which  yields  the  number  of  states  \stf\D  =  8,  is  chosen 
to  generate  state  probability  (p)  vectors  for  the  sym¬ 
bol  sequences. 

The  electronic  circuit  system  is  assumed  to  be  at 
the  nominal  condition  for  the  dissipation  parameter 
1 3  =  0.10,  which  is  selected  as  the  reference  point 
for  calculating  the  anomaly  measure.  The  anomaly 
measure  ,M  is  computed  based  on  two  different  com¬ 
putation  methods  as  discussed  in  Section  5.  Fig.  5 
exhibits  three  plots  of  the  nonnalized  anomaly  mea¬ 
sure  Jt  versus  the  dissipation  parameter  /»',  where 
Ji  is  computed  based  on  different  metrics.  In  each 


Parameter  p 

Fig.  5.  Anomaly  measure  versus  parameter  p. 
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case,  the  reference  point  of  nominal  condition  is  rep¬ 
resented  by  the  parameter  (1  =  0.10.  The  first  plot, 
shown  in  solid  line,  shows  Ji  expressed  as  the  angle 
(in  radians)  between  the  p  vectors  of  the  state  ma¬ 
chines  under  nominal  and  anomalous  conditions,  i.e., 
Ji  =  I—  (Pref j  P)  =  The  remain- 

ing  two  plots,  one  in  dashed  line  and  the  other  in  dot¬ 
ted  line,  show  the  anomaly  measure  expressed  as  the 
if  i  -norm  and  ^-norrn  of  the  difference  between  the 
p  vectors  of  the  state  machines  under  nominal  and 
an  anomalous  conditions,  i.e.,  Ji  =  ||prer  —  p||/i  and 
Ji  =  1 1 pref  —  p||/2,  respectively.  In  each  of  the  three 
plots,  Ji  is  normalized  to  unity  for  better  compari¬ 
son.  All  three  plots  in  Fig.  5  show  gradual  increase 
in  the  anomaly  measure  Ji  for  ji  in  the  approximate 
range  of  0.10-0.25.  At  ji  «  0.25  and  onwards,  Ji 
starts  increasing  at  a  faster  rate  and  finally  saturates 
at  ji  ^  0.29.  The  large  values  of  anomaly  measure  at 
ji  =  0.29  and  beyond  indicate  the  occurrence  of  pe¬ 
riod  reduction  as  seen  in  Figs.  3  and  4.  This  abrupt 
disruption,  preceded  by  gradual  changes,  is  analogous 
to  a  phase  transition  in  the  thermodynamic  sense  [4], 
which  can  also  be  interpreted  as  a  catastrophic  disrup¬ 
tion  in  a  physical  process.  Hence,  observation  of  mod¬ 
est  changes  in  the  anomaly  measure  may  provide  very 
early  warnings  for  a  forthcoming  catastrophic  failure 
as  indicated  by  the  gradual  change  in  the  [1  —  Ji  curve. 

Following  the  steps  (11 )— (15)  of  the  inverse  prob¬ 
lem  in  Section  5.1,  the  state  probability  vector  p  can  be 
obtained  for  the  stationary  behavior  under  the  known 
stimulus.  The  a  priori  information  on  the  anomaly 
measure,  generated  in  the  step  F6  of  the  forward  prob¬ 
lem  in  the  Section  5.1,  can  then  be  used  to  determine 
the  possible  range  in  which  ji  lies.  Solutions  of  the  for¬ 
ward  problem  can  generate  more  information  on  dif¬ 
ferent  ranges  of  [1  under  different  input  stimuli.  Thus, 
the  range  of  the  unknown  parameter  j)  can  be  further 
narrowed  down  by  repeating  this  step  for  other  known 
stimuli  as  reported  earlier  [21],  This  ensemble  of  in¬ 
formation  provides  inputs  for  the  inverse  problem  for 
detecting  anomalies  based  on  the  sensor  data  collected 
in  real  time,  during  the  operation  of  machineries. 

7.  Summary  and  conclusions 

This  paper  presents  a  novel  concept  of  anomaly 
detection  in  complex  systems  based  on  the  tools  of 


Symbolic  Dynamics,  Finite  State  Automata,  and  Pat¬ 
tern  Recognition.  It  is  assumed  that  dynamical  sys¬ 
tems  under  consideration  exhibit  nonlinear  dynami¬ 
cal  behavior  on  two  time  scales.  Anomalies  occur 
on  a  slow  time  scale  that  is  (possibly)  several  or¬ 
ders  of  magnitude  larger  than  the  fast  time  scale  of 
the  system  dynamics.  It  is  also  assumed  that  the  un¬ 
forced  dynamical  system  (i.e.,  in  the  absence  of  ex¬ 
ternal  stimuli)  is  stationary  at  the  fast  time  scale  and 
that  any  non-stationary  behavior  is  observable  only  on 
the  slow  time  scale.  This  concept  of  small  change  de¬ 
tection  in  dynamical  systems  is  elucidated  on  an  ac¬ 
tive  electronic  circuit  representing  the  forced  Duffing 
equation  with  a  slowly  varying  dissipation  parameter. 
The  time-series  data  of  stationary  phase  trajectories 
are  collected  to  create  the  respective  symbolic  dynam¬ 
ics  (i.e.,  symbol  sequences)  using  wavelet  transform. 
The  resulting  state  probability  vector  of  the  transition 
matrix  is  considered  as  the  vector  representation  of 
a  phase  trajectory’s  stationary  behavior.  The  distance 
between  any  two  such  vectors  under  the  same  stimu¬ 
lus  is  the  measure  of  anomaly  that  the  system  has  been 
subjected  to.  This  vector  representation  of  anomalies 
is  more  powerful  than  a  scalar  measure.  The  major 
conclusion  of  this  research  is  that  Symbolic  Dynam¬ 
ics  along  with  the  stimulus-response  methodology  and 
having  a  vector  representation  of  anomaly  is  effective 
for  early  detection  of  small  anomalies. 

The  /J- Markov  machine,  proposed  for  anomaly 
detection,  is  a  suboptimal  approximation  of  the 
e-machine.  It  is  important  that  this  approximation  is 
a  sufficiently  accurate  representation  of  the  nominal 
behavior.  Research  in  this  direction  is  in  progress  and 
the  results  will  be  presented  in  a  forthcoming  publi¬ 
cation.  Further  theoretical  research  is  recommended 
in  the  following  areas: 

•  Separation  of  information-bearing  part  of  the  signal 
from  noise. 

•  Identification  of  a  relevant  submanifold  of  the  phase 
space  and  its  partitioning  to  generate  a  symbol  al¬ 
phabet. 

•  Identification  of  appropriate  wavelet  basis  functions 
for  symbol  generation  and  construction  of  a  map¬ 
ping  from  the  wavelet  space  to  the  symbol  space. 

•  Selection  of  the  minimal  D  for  the  D-Markov  ma¬ 
chine  and  identification  of  the  irreducible  submatrix 
of  the  state  transition  matrix  that  contains  relevant 
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information  on  anomalous  behavior  of  the 
dynamical  system. 
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Appendix  A.  Information  theoretic  quantities 

This  appendix  introduces  the  concepts  of  standard 
information-theoretic  quantities:  entropy  rate,  excess 
entropy  and  statistical  complexity  [11],  which  are 
used  to  establish  the  anomaly  measure  in  Section  5. 

Entropy  rate  (hf)\  The  entropy  rate  of  a  symbol 
string  f/'  is  given  by  the  Shannon  entropy  as  follows: 

hn=  lim  (A.l) 

where  H[L]  =  JflsLes4L  P(s'  )  1°g2Cp0i))  is  the 
Shannon  entropy  of  all  /.-blocks  (i.e.,  symbol  se¬ 
quences  of  length  L)  in  //.  The  limit  is  guaranteed 
to  exist  for  a  stationary  process  [5],  The  entropy  rate 
quantifies  the  irreducible  randomness  in  sequences 
produced  by  a  source:  the  randomness  that  remains 
after  the  correlation  and  the  structures  in  longer  and 
longer  sequence  blocks  are  taken  into  account.  For 
a  symbol  string  ff  represented  as  an  e-machine, 

K=h(sx \sr\. 

Excess  entropy  (E):  The  excess  entropy  of  a  sym¬ 
bol  string  is  defined  as 

OO 

E  =  ^  [h,(L)  -  hlt]  (A. 2) 

L=  I 

where  hfi(L)  =  H[L\—H[L—  1]  is  the  estimate  of  how 
random  the  source  appears  if  only  /.-blocks  in  f/'  are 
considered.  Excess  entropy  measures  how  much  addi¬ 
tional  information  must  be  gained  about  the  sequence 


in  order  to  reveal  the  actual  per-symbol  uncertainty  hu, 
and  thus  measures  difficulty  in  the  prediction  of  the 
process.  Excess  entropy  has  alternate  interpretations 
such  as:  it  is  the  intrinsic  redundancy  in  the  process; 
geometrically  it  is  a  sub-extensive  part  of  H{L)\  and 
it  represents  how  much  historical  information  stored 
in  the  present  is  communicated  to  the  future. 

Statistical  complexity  ( C;, )  [11]:  The  information 
of  the  probability  distribution  of  causal  states,  as 
measured  by  Shannon  entropy,  yields  the  minimum 
average  amount  of  memory  needed  to  predict  future 
configurations.  This  quantity  is  the  statistical  com¬ 
plexity  of  a  symbol  string  //,  defined  by  Crutchfield 
and  Young  [7]  as 

n—  1 

C,,  =  H(i f)  =  -J2  [. Pr(Sk )  log 2Pr{Sk)\  (A.4) 

k= 0 

where  n  is  the  number  of  states  of  the  finite-state  ma¬ 
chine  constructed  from  the  symbol  string  9>.  As  shown 
in  [1 1],  E  ^  Cf,  in  general,  and  C,,  =  E  b  Dhfl. 

Appendix  B.  Finite-type  shift  and  sofic  shift 

This  appendix  very  briefly  introduces  the  concept 
of  shift  spaces  with  emphasis  on  finite  shifts  and  sofic 
shifts  that  respectively  characterize  the  /J- Markov  ma¬ 
chine  and  the  e-machine  described  in  the  Section  4.4. 
The  shift  space  formalism  is  a  systematic  way  to  study 
the  properties  of  the  underlying  grammar,  which  rep¬ 
resent  the  behavior  of  dynamical  systems  encoded 
through  symbolic  dynamics.  The  different  shift  spaces 
provide  increasingly  powerful  classes  of  models  that 
can  be  used  to  represent  the  patterns  in  the  dynamical 
behavior. 

Definition  2.1.  Let  si  be  a  finite  alphabet.  The  full 
si! -shift  is  the  collection  of  all  bi-infinite  sequences  of 
symbols  from  si  and  is  denoted  by: 

siz  =  {x  =  (x,),6Z:  Xj  G  si  MiG  Z}.  (A.5) 

Definition  2.2.  The  shift  map  a  on  the  full  shift  siz 
maps  a  point  x  to  a  point  y = <r(x)  whose  z'th  coordinate 
is  y,  =  xi+i . 

A  block  is  a  finite  sequence  of  symbols  over  .si.  Let 
x  G  siz  and  w  be  a  block  over  si.  Then  w  occurs  in  x 
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if  3  indices  i  and  j  such  that  w  =  X|;  /]  =  x,xi+ 1  ■  •  ■  Xj . 
Note  that  the  empty  block  g  occurs  in  every  x. 

Let  (W  be  a  collection  of  blocks,  i.e.,  finite  sequences 
of  symbols  over  s4 .  Let  x  €  ,r/z  and  w  be  a  block  over 

.  Then  w  occurs  in  x  if  3  indices  i  and  j  such  that 
w  =X[,  /|  =  x,xJ+ 1  ■  ■  -Xj.  For  any  such  .4 ,  let  us  define 
Xy  to  be  the  subset  of  sequences  in  s4z ,  which  do  not 
contain  any  block  in  .¥ . 

Definition  2.3.  A  shift  space  is  a  subset  A  of  a  full 
shift  ,r/z  such  that  X  =  Xy  for  some  collection  ■¥  of 
forbidden  blocks  over  s4 . 

For  a  given  shift  space,  the  collection  3F  is  at  most 
countable  (i.e.,  finite  or  countably  infinite)  and  is 
non-unique  (i.e.,  there  may  be  many  such  J^’s  de¬ 
scribing  the  shift  space).  As  subshifts  of  full  shifts, 
these  spaces  share  a  common  feature  called  shift  in¬ 
variance.  Since  the  constraints  on  points  are  given  in 
terms  of  forbidden  blocks  alone  and  do  not  involve 
the  coordinate  at  which  a  block  might  be  forbidden,  it 
follows  that  if x  elf,  then  so  are  its  shifts  cr(x)  and 
cr_1(x).  Therefore  c>(X:y  )  =  Xy. ,  which  is  a  necessary 
condition  for  a  subset  of  ■rfz  to  be  a  shift  space.  This 
property  introduces  the  concept  of  shift  dynamical 
systems. 

Definition  2.4.  Let  A  be  a  shift  space  and  oy  :  A  — > 
A  be  the  shift  map.  Then  (A,  ax )  is  known  as  a  shift 
dynamical  system. 

The  shift  dynamical  system  mirrors  the  dynamics 
of  the  original  dynamical  system  from  which  it  is  gen¬ 
erated  (by  symbolic  dynamics).  Several  examples  of 
shift  spaces  are  given  in  [15]. 

Rather  than  describing  a  shift  space  by  specify¬ 
ing  the  forbidden  blocks,  it  can  also  be  specified  by 
allowed  blocks.  This  leads  to  the  notion  of  a  language 
of  a  shift. 

Definition  2.5.  Let  A  be  a  subset  of  a  full  shift,  and 
let  ,(%„( A)  denote  the  set  of  all  «-blocks  (i.e.,  blocks 
of  length  n )  that  occur  in  A.  The  language  of  the  shift 
space  A  is  defined  as: 

OO 

£(A)  =  (J  m,(X).  (A.6) 

n= 0 


Sliding  block  codes :  Let  A  be  a  shift  space  over  ,r4 , 
then  x  G  A  can  be  transformed  into  a  new  sequence  y = 

•  •  •  y-iyoyi  ■  ■  ■  over  another  alphabet  °U  as  follows. 

Fix  integers  m  and  n  such  that  —m  4  n.  To  compute 
y,  of  the  transformed  sequence,  we  use  a  function  (l> 
that  depends  on  the  “window”  of  coordinates  ofx  from 
i  —  m  to  i  +  n.  Here  (l> :  341  m ,  „+ 1  (A )  >  °U  is  a  fixed 

block  map,  called  a  (m  +  n  +  1  )-block  map  from  the 
allowed  (m  +  n  +  1  )-blocks  in  A  to  symbols  in  °U. 
Therefore, 

yi  —  I  I  '  '  ‘  X  i  ■  n  )  ^(Vj i  m.i  \  ti\) ■  (  A.7  ) 

Definition  2.6.  Let  $  be  a  block  map  as  defined  in 
Eq.  (A.7).  Then  the  map  cf>: A  — >  (7/)z  defined  by 
y  =  with  y,  given  by  Eq.  (A.7)  is  called  the 
sliding  block  code  with  memory  m  and  anticipation  n 
induced  by  <P. 

Definition  2.7.  Let  A  and  Y  be  shift  spaces,  and  4>  : 
X  — >  Y  be  a  sliding  block  code. 

•  If  (f> :  X  — >  Y  is  onto,  then  0  is  called  a  factor  code 
from  A  onto  Y. 

•  If  (j) :  X  — >  Y  is  one-to-one,  then  ([>  is  called  an 
embedding  of  A  into  Y . 

•  If  (/) :  X  — >  Y  has  a  inverse  (i.e.,  3  a  sliding  block 
code  i//  :Y  — >  A  such  that  i p(<f>(x))  =  x  Vx  €  A  and 

=  y  £  k),  then  </)  is  called  a  conjugacy 
from  A  to  Y . 

If  3  a  conjugacy  from  A  to  Y,  then  Y  can  be  viewed 
as  a  copy  of  A,  sharing  all  properties  of  A.  Therefore, 
a  conjugacy  is  often  called  topological  conjugacy  in 
literature. 

Finite-type  shifts :  We  now  introduce  the  concept  of 
finite-type  shift  that  is  the  structure  of  the  shift  space 
in  the  D-Markov  machine  proposed  in  the  Section  4.2. 

Definition  2.8.  K finite-type  shift  is  a  shift  space  that 
can  be  described  by  a  finite  collection  of  forbidden 
blocks  (i.e.,  A  having  the  form  Ajr  for  some  finite  set 
i ¥  of  blocks). 

An  example  of  a  finite  shift  is  the  golden  mean  shift, 
where  the  alphabet  is  S  =  {0, 1}  and  the  forbidden  set 
:W  =  {11}.  That  is,  A  =  Xy  is  the  set  of  all  binary 
sequences  with  no  two  consecutive  l’s. 
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Definition  2.9.  A  finite-type  shift  is  /V/-stcp  or  has 
memory  M  if  it  can  be  described  by  a  collection  of 
forbidden  blocks  all  of  which  have  length  M  +  1 . 

The  properties  of  a  finite-type  shift  are  listed  below: 

•  If  A  is  a  finite-type  shift,  then  3M  ^  0  such  that  A 
is  M- step. 

•  The  language  of  the  finite-type  shift  is  characterized 
by  the  property  that  if  two  words  overlap,  then  they 
can  be  glued  together  along  their  overlap  to  from 
another  word  in  the  language.  Thus,  a  shift  space 
X  is  an  M- step  finite-type  shift  iff  whenever  uv, 
vw  €  30 (X )  and  |u|  ^  M,  then  uvw  £  30 (X). 

•  A  shift  space  that  is  conjugate  to  a  finite-type  shift 
is  itself  a  finite-type  shift. 

•  A  finite-type  shift  can  be  represented  by  a  finite, 
directed  graph  and  produces  the  collection  of  all 
bi-infinite  walks  (i.e.  sequence  of  edges)  on  the 
graph. 

Sofic  shifts :  The  sofic  shift  is  the  structure  of  the 
shift  space  in  the  e-machines  [7,20]  in  Section  4.1. 
Let  us  label  the  edges  of  a  graph  with  symbols  from 
an  alphabet  where  two  or  more  edges  are  allowed 
to  have  the  same  label.  Every  bi-infinite  walk  on  the 
graph  yields  a  point  in  .c/z  by  reading  the  labels  of  its 
edges,  and  the  set  of  all  such  points  is  called  a  sofic 
shift. 

Definition  2.10.  A  graph  G  consists  of  a  finite  set 
ir  =  Y~{G)  of  vertices  together  with  a  finite  set  &  = 
S(G)  of  edges.  Each  edge  e  £  Si G )  starts  at  a  vertex 
denoted  by  /'(e)  £  A~(G)  and  tenninates  at  a  vertex 
t(e)£  t  (G)  (which  can  be  the  same  as  /(e)).  There 
may  be  more  that  one  edge  between  a  given  initial 
state  and  terminal  state;  a  set  of  such  edges  is  called 
a  set  of  multiple  edges.  An  edge  e  with  /'(e)  =  t(e)  is 
called  a  self-loop. 

Definition  2.11.  A  labelled  graph  3#  is  a  pair  (G,  3/J), 
where  G  is  a  graph  with  edge  set  $ ,  and  3£ :  8  — ^  .?/ 
assigns  a  label  -£(e)  to  each  edge  e  of  G  from  the 
finite  alphabet  s£ .  The  underlying  graph  of  £  is  G. 

Definition  2.12.  A  subset  A  of  a  full  shift  is  a 
sofic  shift  if  X  =  X<$  for  some  labelled  graph  £ .  A 


presentation  of  a  sofic  shift  A  is  a  labelled  graph  £ 
for  which  A^  =  A. 

An  example  of  a  sofic  shift  is  the  even  shift,  which  is 
the  set  of  all  binary  sequences  with  only  even  number 
of  0’s  between  any  two  l’s.  That  is,  the  forbidden  set 
OX  is  the  collection  {102"+l :  n  ^  0}. 

Some  of  the  salient  characterization  of  sofic  shifts 
are  presented  below  [15]: 

•  Every  finite-type  shift  qualifies  as  a  sofic  shift. 

•  A  shift  space  is  sofic  iff  it  is  a  factor  of  a  finite-type 
shift. 

•  The  class  of  sofic  shifts  is  the  smallest  collection 
of  shifts  spaces  that  contains  all  finite-type  shifts 
and  also  contains  all  factors  of  each  space  in  the 
collection. 

•  A  sofic  shift  that  does  not  have  finite-type  subshifts 
is  called  a  strictly  sofic.  For  example,  the  even  shift 
is  strictly  sofic  [15]. 

•  A  factor  of  a  sofic  shift  is  a  sofic  shift. 

•  A  shift  space  conjugate  to  a  sofic  shift  is  itself  sofic. 

•  A  distinction  between  finite-type  shifts  and  sofic 
shifts  can  be  characterized  in  terms  of  the  memory. 
While  finite-type  shifts  use  finit ^-length  memory, 
sofic  shifts  require  finite  amount  of  memory.  In 
contrast,  context-free  shifts  require  infinite  amount 
of  memory  [12]. 
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Abstract 

Symbolic  time  series  analysis  (STSA)  of  complex  systems  for  anomaly  detection  has  been  recently  introduced  in 
literature.  An  important  feature  of  the  STSA  method  is  extraction  of  relevant  information,  imbedded  in  the  measured  time 
series  data,  to  generate  symbol  sequences.  This  paper  presents  a  wavelet-based  partitioning  approach  for  symbol 
generation,  instead  of  the  currently  practiced  method  of  phase-space  partitioning.  Various  aspects  of  the  proposed 
technique,  such  as  wavelet  selection,  noise  mitigation,  and  robustness  to  spurious  disturbances,  are  discussed.  The  wavelet- 
based  partitioning  in  STSA  is  experimentally  validated  on  laboratory  apparatuses  for  anomaly/damage  detection,  its 
efficacy  is  investigated  by  comparison  with  phase-space  partitioning. 

©  2006  Elsevier  B.V.  All  rights  reserved. 
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1.  Introduction 

The  concept  of  symbolic  time  series  analysis 
(STSA)  has  been  recently  proposed  for  anomaly 
detection  in  complex  systems  [1],  Several  case 
studies  [2-5]  in  anomaly  detection  show  that  STSA 
can  be  more  effective  than  existing  pattern  recogni¬ 
tion  techniques  (e.g.,  principal  component  analysis 
and  neural  networks).  The  STSA  method  has  also 
been  demonstrated  for  fault  detection  in  electro¬ 
mechanical  systems,  such  as  three-phase  induction 
motors  [6]  and  helical  gearbox  in  rotorcraft  [7]. 
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A  crucial  step  in  STSA  is  extraction  of  relevant 
information,  imbedded  in  the  measured  time  series 
data,  to  generate  symbol  sequences.  Symbol  gen¬ 
eration  requires  partitioning  of  the  data  space  to 
obtain  the  symbol  sequences  [8,9].  Various  parti¬ 
tioning  techniques  have  been  suggested  in  literature 
for  symbol  generation,  which  include  variance- 
based  [10],  entropy-based  [11],  and  hierarchical 
clustering  [12]  methods.  A  survey  of  clustering 
techniques  is  provided  in  [13].  In  addition  to  these 
methods,  another  scheme  of  partitioning,  based  on 
symbolic  false  nearest  neighbors  ( SFNN ),  was 
reported  by  Kennel  and  Buhl  [14],  The  objective 
of  SFNN  partitioning  is  to  ensure  that  points 
that  are  close  to  each  other  in  the  symbol  space  are 
also  close  to  each  other  in  the  phase  space. 
Partitions  that  yield  a  smaller  proportion  of 
SFNN  are  considered  optimal.  However,  this 
partitioning  method  may  become  computationally 
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very  expensive  if  the  dimension  of  the  phase  space  is 
large  or  if  the  data  set  is  contaminated  by  noise, 
since  noise  induces  false  symbols. 

Partitioning  of  wavelet-transformed  data  poten¬ 
tially  alleviates  the  above-mentioned  shortcomings 
and  is  particularly  effective  with  noisy  data  from 
high-dimensional  dynamical  systems.  Usage  of 
wavelet  transform  for  symbolization  has  been 
recently  introduced  by  the  authors  [1,15].  This 
paper  elaborates  the  concept  of  wavelet-based 
partitioning  for  STSA  and  its  major  features  are 
delineated  below. 

•  Selection  of  the  wavelet  basis  and  scale  range. 

•  Noise  mitigation  in  the  measured  time  series  data 
prior  to  symbol  generation. 

•  Robustness  of  extracted  information  from  sym¬ 
bol  sequences. 

•  Enhancement  of  computational  efficiency  for 
anomaly  detection. 

•  Validation  of  the  STSA  method  on  laboratory 
apparatuses  for  anomaly/damage  detection. 

The  paper  is  organized  into  six  sections  including 
the  present  section.  Section  2  focuses  on  key 
technical  aspects  of  wavelet  analysis  such  as 
selection  of  wavelet  basis  and  scales.  Section  3 
presents  the  maximum  entropy  method  of  partition¬ 
ing  the  space  of  wavelet  coefficients.  Section  4 
elucidates  the  underlying  principles  of  wavelet- 
based  partitioning  via  simulation  examples.  Section 
5  presents  experimental  results  on  laboratory 
apparatuses  to  demonstrate  efficacy  of  wavelet- 
based  STSA  for  anomaly  detection.  Section  6 
summarizes  the  paper  and  makes  major  conclusions 
along  with  recommendations  for  future  research. 

2.  Wavelet  analysis  of  time  series  data 

This  section  presents  generation  of  wavelet 
coefficients  from  measured  time  series  data,  and 
their  arrangement  for  symbol  generation.  Specifi¬ 
cally,  issues  of  wavelet  basis  and  scale  range 
selection  are  addressed. 

Preprocessing  of  time  series  data  is  often  neces¬ 
sary  for  extraction  of  pertinent  information.  Fourier 
analysis  is  sufficient  if  the  signal  to  be  analyzed  is 
stationary  and  if  the  time  period  is  accurately 
known.  Flowever,  Fourier  analysis  may  not  be 
appropriate  if  the  signal  has  non-stationary  char¬ 
acteristics  such  as  drifts,  abrupt  asynchronous 
changes,  and  frequency  trends.  Wavelet  analysis 


alleviates  these  difficulties  via  adaptive  usage  of  long 
windows  for  retrieving  low-frequency  information 
and  short  windows  for  high-frequency  information 
[16,17].  The  ability  to  perform  flexible  localized 
analysis  is  one  of  the  striking  features  of  wavelet 
transform. 

In  multi-resolution  analysis  ( MRA )  of  wavelet 
transform,  a  continuous  signal  /  e  H,  where  H  is  a 
Hilbert  space,  is  decomposed  as  a  linear  combina¬ 
tion  of  time  translations  of  scaled  versions  of  a 
suitably  chosen  scaling  function  cf>(t )  and  the  derived 
wavelet  function  i j/(t).  Let  the  sequence  {4>jk}  belong 
to  another  Hilbert  space  M  with  a  countable 
measure,  where  the  scale  ^  =  T  and  time  translation 
t  =  2~jk.  If  the  sequence  {<f>jk}  is  a  frame  for  the 
Hilbert  space  H  with  a  frame  representation 
operator  L,  then  there  are  positive  real  scalars  A 
and  B  such  that 

^I[/'IIh^IIl/IIm^5II/'IIh  V/e  H,  (1) 

where  1/  =  {(/,  ^/c>}  and  ||l/j|M  is  an  appropriate 
norm,  e.g.,  \\lf\\M  =  yJ'Ej'LkW’ <MI:  is  a  candi¬ 
date  norm;  and  {x,y)  is  the  inner  product  of  x  and  v, 
both  belonging  to  H. 

The  above  relationship  is  a  norm  equivalence 
and  represents  the  degree  of  coherence  of  the  signal 
/ with  respect  to  the  frame  set  of  scaling  functions;  it 
may  be  interpreted  as  enforcing  an  approximate 
energy  transfer  between  the  domains  H  and  L(H). 
In  other  words,  for  all  signals  f  e  H,  a  scaled 
amount  of  energy  is  distributed  in  the  coefficient 
domain  where  the  scale  factor  lies  between  A  and  B 
[16].  However,  the  energy  distribution  is  dependent 
on  the  signal’s  degree  of  coherence  with  the  under¬ 
lying  frame  {cj)jk}.  For  a  signal /,  which  is  coherent 
with  respect  to  the  frame  { <j)jk } ,  norm  equivalence  in 
the  frame  representation  necessarily  implies  that  a 
few  coefficients  contain  most  of  the  signal  energy 
and  hence  have  relatively  large  magnitudes.  Simi¬ 
larly,  pure  noise  signal  w  being  incoherent  with 
respect  to  the  set  {4>jk},  must  have  a  frame 
representation  in  which  the  noise  energy  is  spread 
out  over  a  large  number  of  coefficients.  Conse¬ 
quently,  these  coefficients  have  a  relatively  small 
magnitude  [17]. 

Let  /  be  a  noise  corrupted  version  of  the  original 
signal  /  expressed  as 

f=f  +  <jw,  (2) 

where  w  is  additive  white  gaussian  noise  with  zero 
mean  and  unit  variance  and  a  is  the  noise  level. 
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Then,  the  inner  product  of /  and  (pj  k  is  obtained  as 

(/> 4>j,k)  =  ■  (3) 

signal  part  noise  part 

The  noise  part  in  Eq.  (3)  may  further  be  reduced  if 
the  scales  over  which  coefficients  are  obtained  are 
properly  chosen. 

For  every  wavelet,  there  exists  a  certain  frequency 
called  the  center  frequency  Fc  that  has  the 
maximum  modulus  in  the  Fourier  transform  of 
the  wavelet  [18].  The  pseudo-frequency  /p  of  the 
wavelet  at  a  particular  scale  a  is  given  by  the 
following  formula  [18,19]: 


where  At  is  the  sampling  interval.  Fig.  1  depicts  the 
center  frequency  associated  with  the  Daubechies  4 
wavelet  ‘db4’  [16,20], 

The  power  spectral  density  (PSD)  of  the  signal 
provides  the  information  about  the  frequency 
content  of  the  signal.  This  information  along  with 
Eq.  (4)  can  be  used  for  scale  selection.  The 
procedure  of  selecting  the  scales  is  summarized 
below: 

•  Identification  of  the  frequencies  of  interest 
through  PSD  analysis  of  time  series  data. 

•  Substitution  of  the  above  frequencies  in  place  of 
/p  in  Eq.  (4)  to  obtain  the  respective  scales  in 
terms  of  the  known  parameters  Fc  and  At. 

The  wavelet  coefficients  of  the  signal  are  sig¬ 
nificantly  large  when  the  pseudo-frequency  /  of  the 
wavelet  corresponds  to  the  locally  dominant  fre¬ 
quencies  in  the  underlying  signal.  Example  1  in 
Section  4  illustrates  how  the  choice  of  wavelet  and 
scale  affect  the  coefficients.  Examples  2  and  3 


Fig.  1.  Center  frequency  approximation  for  wavelet  db4. 


illustrate  noise  suppression  and  robustness.  Three 
experimental  studies  in  Section  5  illustrate  enhance¬ 
ment  of  anomaly/damage  detection  using  STSA  and 
make  comparisons  of  wavelet-based  partitioning 
and  SFNN  phase-space  partitioning  in  this  regard. 

Upon  selection  of  the  wavelet  basis  and  scale 
range,  the  wavelet  coefficients  are  obtained.  These 
coefficients  are  stacked  at  selected  time-shift  posi¬ 
tions,  starting  with  the  smallest  value  of  scale  and 
ending  with  its  largest  value  and  then  back  from  the 
largest  value  to  the  smallest  value  of  the  scale  at  the 
next  instant  of  time  shift.  In  the  sequel,  this  one¬ 
dimensional  array  of  arranged  wavelet  coefficients  is 
called  the  scale  series  data,  which  is  structurally 
similar  to  time  series  data  in  the  phase  space.  For 
symbol  generation,  the  scale  series  data  can  be 
handled  in  a  similar  way  as  time  series  data. 

3.  Symbolization  of  scale  series  data 

This  section  presents  symbolization  of  wavelet 
coefficients  by  maximum  entropy  partitioning.  The 
scale  series  data  are  partitioned  to  construct  the 
symbol  alphabet  and  to  generate  symbol  sequences. 

In  the  wavelet-based  partitioning  scheme,  as 
introduced  in  an  earlier  publication  [1],  the  max¬ 
imum  and  minimum  of  the  scale  series  are  evaluated 
and  the  ordinates  between  the  maximum  and 
minimum  are  divided  into  equal-sized  regions. 
These  regions  are  mutually  disjoint  and  thus  form 
a  partition.  Each  region  is  then  labelled  with  one 
symbol  from  the  alphabet.  If  the  data  point  lies  in  a 
particular  region,  it  is  coded  with  the  symbol 
associated  with  that  region.  Thus,  a  sequence  of 
symbols  is  created  from  a  given  sequence  of  scale 
series  data.  This  type  of  partitioning  is  called 
uniform  partitioning  in  the  sequel.  The  left-hand 
plot  in  Fig.  2  depicts  uniform  partitioning  of  noise 
contaminated  signal  sin(27i?)  with  alphabet  size 
|T|  =4.  Note  that  the  partition  segments  are  of 
equal  size. 

Intuitively,  it  is  more  reasonable  if  the  informa¬ 
tion-rich  regions  of  the  data  set  are  partitioned  finer 
and  those  with  sparse  information  are  partitioned 
coarser.  To  achieve  this  objective,  a  partitioning 
method  is  adopted  such  that  the  entropy  of  the 
generated  symbol  sequence  is  maximized  [15]. 
Maximum  entropy  partitioning  is  abbreviated  as 
ME  partitioning  in  the  sequel.  The  procedure  for 
obtaining  an  ME  partition  is  described  below. 

Let  N  be  the  length  of  the  data  set  and  T|  be  the 
size  of  the  symbol  alphabet  (i.e.,  the  number  of 
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Uniform  Partitioning  Maximum  Entropy  Partitioning 


Fig.  2.  Examples  of  uniform  and  ME  partitioning  with  \I\  =  4. 


disjoint  elements  in  the  partition).  The  data  set  is 
sorted  in  ascending  order.  Starting  from  the  first 
point  in  the  sorted  data,  every  consecutive  data 
segment  of  length  L-V/jT|J  forms  a  distinct  element 
of  the  partition,  where  |_-*J  represents  the  greatest 
integer  less  than  or  equal  to  v. 

The  right-hand  plot  in  Fig.  2  shows  ME  partition¬ 
ing  for  the  noise-contaminated  signal  sin(27i?)  with 
\E\  —  4.  As  expected,  the  size  of  the  partitions  are  not 
equal,  but  the  probabilities  of  the  symbols  are  equal. 
Variations  in  data  patterns  are  more  likely  to  be 
reflected  in  the  symbol  sequence  obtained  under  ME 
partitioning  than  other  partitioning. 

The  choice  of  the  alphabet  size  T  plays  a  crucial 
role  in  STSA.  For  example,  a  small  value  of  |T|  may 
prove  inadequate  for  capturing  the  characteristics  of 
the  scale  series  data.  On  the  other  hand,  a  large 
value  of  \E\  may  lead  to  redundancy  and  waste  of 
computational  resources. 

Selection  of  the  alphabet  size  T  is  an  area  of 
active  research;  an  entropy-based  approach  has 
been  adopted  for  selecting  |T|  in  this  paper.  Let 
H(k )  denote  the  Shannon  entropy  of  the  symbol 
sequence  obtained  by  partitioning  the  data  set  with 
k  symbols: 

i—k 

H(k)  =  -  J2pi  l°SlPr  (5) 

i=  1 

where  pf  represents  the  probability  of  occurrence  of 
the  symbol  cr, .  Note  that  H(\)  =  0  because pt  =  0  or 
1  with  i  —  1 .  If  the  underlying  data  set  has  sufficient 
information  content,  then  the  entropy  achieved 


under  ME  partitioning  would  be  log2(k),  which 
corresponds  to  the  uniform  distribution.  We  define 
a  quantity  /;(■)  to  represent  the  change  in  entropy 
with  respect  to  the  number  |T|  of  symbols  as 

h{k)  =  H(k)  -H(k-\)  VL=s2.  (6) 

The  algorithm  for  alphabet  size  selection  is  given 
below. 

Step  1:  Set  k  =  2.  Choose  a  threshold  £*,  where 
0  <  £/,  <^  1 . 

Step  2:  Sort  the  data  set  (of  length  TV)  in  the 
ascending  order. 

Step  3:  Every  consecutive  segment  of  length 
L N /k\  in  the  sorted  data  set  (of  length  TV)  forms  a 
distinct  element  of  the  partition. 

Step  4:  Convert  the  raw  data  into  a  symbol 
sequence  with  the  partitions  obtained  in  Step  3.  If 
the  data  point  lies  within  or  on  the  lower  bound  of  a 
partition,  it  is  coded  with  the  symbol  associated 
with  that  partition. 

Step  5:  Compute  the  symbol  probabilities  /;,. 
i  =  1,2,...,  k. 

Step  6:  Compute  H{k)  =  —  Y2iAPi  l°§2/h  and 
h(k)  —  H(k)  —  H(k  —  1). 

Step  7:  If  h(k)  <  £/,,  then  exit;  else  increment  k  by  1 
and  go  to  Step  3. 

In  general,  a  small  £/,  leads  to  a  large  size  of  the 
symbol  alphabet,  resulting  in  increased  computa¬ 
tion.  Also,  a  larger  alphabet  makes  the  partitioning 
finer.  This  might  increase  the  probability  of  false 
symbols  being  induced  by  noise.  On  the  other  hand, 
a  large  £/,  leads  to  a  small  alphabet  size  that 
may  prove  inadequate  for  capturing  the  pertinent 
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information.  Hence,  there  is  a  trade-off  between 
accuracy  and  computational  speed  when  e /,  is 
chosen.  The  variance  of  the  noise  process  associated 
with  the  signal  may  serve  as  a  guideline  for  selection 
of  eh. 

4.  Validation  of  wavelet-based  partitioning 

This  section  presents  simulation  cases  to  validate 
symbolization  of  measured  time  series  data  via 
partitioning  of  the  wavelet  coefficients.  The  under¬ 
lying  concepts  are  illustrated  by  three  examples. 

4.1.  Example  1:  choice  of  wavelet  parameters 

This  example  illustrates  how  the  choice  of  wavelet 
basis  and  scale  range  affects  the  coefficients  that,  in 
turn,  determine  symbol  generation  for  anomaly 
detection  [1].  Let  us  consider  the  following  sinusoi¬ 
dal  signal: 

y(t)  =  cos(2nt)  V?  e  [—5,  +5].  (7) 

The  frequency  of  y(t)  in  Eq.  (7)  is  1.00  Hz.  The 
Gaussian  wavelet  9  (‘gaus9’)  (see  [20,  p.  74]) 
matches  the  shape  of  y(t)  well,  as  seen  in  Fig.  3 
that  compares  an  appropriately  scaled  and  trans¬ 
lated  version  of  the  ‘gaus9’  wavelet  with  the 
signal  y(t). 

To  demonstrate  the  impact  of  the  chosen  wavelet 
parameters  on  the  coefficients,  the  wavelet  basis 
‘dbl’  is  also  considered  for  comparison  purposes. 
The  signal  y(t)  is  sampled  at  100  Hz  (i.e.,  the 
sampling  interval  At  =  0.01s).  The  wavelet  coeffi¬ 
cients  of  the  signal  y(t)  are  obtained  for  various 
scales  with  both  wavelets,  ‘gaus9’  and  ‘dbl’.  The 


Fig.  3.  Comparison  of  the  wavelet  basis  ‘gaus9’  and  the  signal. 


norm  of  the  coefficients  corresponding  to  each  scale 
and  the  pseudo-frequencies  of  the  wavelet  corre¬ 
sponding  to  the  chosen  scales  are  calculated.  Fig.  4 
shows  the  plot  of  the  norm  of  coefficients  and  the 
pseudo-frequencies  of  the  wavelet. 

It  is  observed  in  Fig.  4  that,  for  both  wavelets 
‘gaus9’  and  ‘dbl’,  the  maximum  of  the  norm  is 
obtained  at  /  ~  1.00  Hz.  In  fact,  it  is  exactly  at 
1.00  Hz  for  ‘gaus9’.  Furthermore,  the  value  of  the 
peak  norm  achieved  with  ‘gaus9’  is  appreciably 
greater  than  that  with  wavelet  ‘dbl’.  In  other  words, 
the  coefficients  obtained  with  ‘gaus9’  are  more 
significant  than  those  obtained  with  ‘dbl’.  Another 
observation  is  that  the  norm  curve  for  ‘gaus9’  shows 
a  greater  rate  of  decay  across  pseudo-frequencies 
than  that  of  ‘dbl’.  More  energy  is  concentrated  in  a 
narrow  band  frequencies  around  1.00  Hz  in  the  case 
of  ‘gaus9’.  These  observations  imply  that  high 
energy  compaction  can  be  achieved  with  fewer 
coefficients  if  the  wavelet  and  the  scales  are  chosen 
as  stated  in  Section  2.  A  favorable  implication  of 
fewer  coefficients  is  fewer  number  of  symbols  for 
analysis  and  hence  an  improvement  in  computa¬ 
tional  efficiency. 

4.2.  Example  2:  noise  suppression 

This  example  demonstrates  how  noise  suppres¬ 
sion  is  achieved  with  wavelets.  Let  the  signal  y(t)  in 
Eq.  (7)  be  corrupted  with  additive  zero-mean  white 
Gaussian  noise  w(t), 

y(t)  =  y(t )  +  <jw(t).  (8) 

A  common  measure  of  noise  in  a  noise-corrupted 
signal  is  the  signal-to-noise  ratio  (SNR)  that  is 


Norm  of  Coefficients  Vs  Pseudo-Frequency 


Fig.  4.  Coefficient  norm  and  pseudo-frequency  for  different 
wavelets. 
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Fig.  5.  Signal  and  noise  profiles  at  a  =  0.05. 


Fig.  6.  Signal  and  noise  profiles  at  a  =  0.10. 


defined  as: 


SNR^J%,  (9) 

Ik  w|lH 

where  y  and  w  are  functions  of  time.  Similar  to 
Eq.  (9),  the  SNR  in  the  wavelet  domain  is  defined  as 


SNR  = 


11411m 

MIm’ 


(10) 


where  L y  and  L w,  which  belong  to  the  Hilbert  space 
M  (see  Section  2),  represent  the  wavelet  coefficients 
of  the  signal  y  and  the  noise  w. 

Numerical  experiments  have  been  performed  with 
a  e  {0.05,0.1}.  The  signal  is  sampled  at  100 Hz  (i.e., 
At  —  0.01s).  The  scales  are  determined  following 
Eq.  (4),  such  that  the  pseudo-frequency  of  the 
wavelet  matches  the  frequency  of  the  signal.  Fig.  5 
depicts  the  time  domain  plot  (left  plate)  and 
coefficient  plot  (right  plate)  of  the  signal  y  and 
white  Gaussian  noise  having  standard  deviation 
a  —  0.05.  Similarly,  Fig.  6  depicts  the  time  domain 
plot  (left  plate)  and  coefficient  plot  (right  plate) 
of  the  signal  y  and  white  Gaussian  noise  having 
standard  deviation  er=JI10.  Table  1  lists  the 
values  of  SNR  and  SNR,  averaged  over  20 
simulation  runs. 

Both  Figs.  5  and  6  show  that  SNR  is  significantly 
larger  than  SNR.  That  is,  the  wavelet-transformed 
signal  is  significantly  de-noised  relative  to  the  time 
domain  signal.  This  is  expected  because  the  noise  is 
incoherent  with  the  wavelet  while  the  signal  enjoys  a 
great  degree  of  coherence  with  the  wavelet.  Thus, 
symbols  generated  from  wavelet  coefficients  would 
reflect  the  characteristics  of  the  signal  with  more 
fidelity  than  those  obtained  with  time  domain 
signals. 


Table  1 
SNR  values 


a  =  0.05 

a  =  0.1 

SNR 

191.55 

50.89 

SNR 

25195 

4281.5 

4.3.  Example  3:  robustness  of  symbol  probability 
vector 

The  symbolization  scheme  is  developed  to  en¬ 
hance  real-time  detection  of  slowly  varying  anoma¬ 
lies  in  dynamical  systems  [1].  Of  critical  importance 
is  the  symbol  probability  vector  p  whose  elements 
denote  the  probability  of  occurrence  of  individual 
symbols  in  the  symbol  sequence.  The  vector  p  must 
be  robust  relative  to  measurement  noise  and 
spurious  disturbances  and,  at  the  same  time,  be 
sensitive  enough  to  detect  small  slowly  varying 
anomalies  from  the  observed  data  set.  A  distortion 
measure  for  the  symbol  probability  vector  is 
introduced  below 

&t-\\Pt-Pt\\u  (11) 

where  the  subscript  t  denotes  that  the  probability 
vectors  correspond  to  symbols  generated  from  time 
domain  signals;  and  11*11!  is  the  sum  of  the  absolute 
values  of  the  elements  of  the  vector  *.  The  vector  p„ 
with  ||/i,||  ]  =  1,  corresponds  to  the  uncorrupted 
signal  and  pt  corresponds  to  the  corrupted  signal. 
Similar  to  Eq.  (11),  distortion  measure  in  the 
wavelet  scale  domain  is  defined  as 

<5s-ILps-Alli>  (12) 

where  the  subscript  s  denotes  that  the  probability 
vectors  correspond  to  symbols  generated  from 
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wavelet  scale  domain  signals  (i.e.,  scale  series  data). 
Therefore,  lower  is  the  distortion  ratio,  closer  is  the 
probabilistic  representation  of  the  corrupted  signal 
to  that  of  the  uncorrupted  signal,  i.e.,  greater  is  the 
robustness  to  noise  and  spurious  disturbances. 

The  partitions  are  obtained,  in  case  of  time 
domain,  by  employing  the  maximum  entropy 
criterion  on  the  time  series  data  of  the  signal.  In 
the  wavelet  domain,  the  partitions  are  obtained  with 
the  corresponding  scale  series  data,  as  defined  in 
Section  2.  In  both  time  domain  and  wavelet  domain, 
the  probability  vectors  p  and  p  are  computed  with 
the  same  partitions  for  the  uncorrupted  and 
corrupted  signals. 

The  symbol  alphabet  size  |T|  is  chosen  to  be  4  in 
this  example.  The  partitions  are  obtained  as 
mentioned  before  for  the  signal  y  and  its  transform, 
i.e.,  the  coefficient  vector  IL y.  Table  2  lists  the  values 
of  distortion  ratios  8,  and  ds,  averaged  over  20 
simulation  runs. 

It  is  seen  that  distortion  measures  are  far  smaller 
in  the  wavelet  scale  domain  than  those  in  the  time 
domain.  This  observation  implies  that  the  symbol 
probabilities  are  significantly  more  robust  to  mea¬ 
surement  noise  and  spurious  disturbances  in  the 
wavelet  domain  than  in  the  time  domain.  Hence,  it 
may  be  inferred  that  symbols  generated  from  the 
wavelet  coefficients  would  be  better  for  anomaly 
detection  as  the  effects  of  noise  to  induce  errors  in 
the  symbol  probabilities  are  significantly  mitigated. 

5.  Experimental  results  on  anomaly  detection 

This  section  evaluates  the  performance  of  wave¬ 
let-based  partitioning  in  STSA  for  anomaly/damage 
detection.  This  is  demonstrated  via  experimentation 
on  the  following  laboratory  apparatuses: 

•  Anomaly  detection  on  a  nonlinear  electronic 
system  apparatus  [1]. 

•  Structural  damage  detection  on  a  mechanical 
vibration  system  apparatus  [3], 

•  Damage  detection  in  polycrystalline  alloys  on  a 
fatigue  testing  apparatus  [5], 

Table  2 

Distortion  ratios 

a  =  0.05  a  =  0. 1 

<5,  0.040  0.054 

<5,  0.006  0.010 


5.1.  Anomaly  detection  in  nonlinear  systems 

This  example  demonstrates  efficacy  of  the  STSA 
method  for  anomaly  detection  in  nonlinear  systems. 
Experiments  have  been  conducted  on  a  laboratory 
apparatus  [1]  that  emulates  the  forced  Duffing 
equation  [21],  modelled  as 

^  +  y(t )  +  y3(t)  =  A  cos(Qt),  (13) 

where  the  dissipation  parameter  /?  varies  slowly  with 
respect  to  the  response  of  the  dynamical  system;  p  = 
0.1  represents  the  nominal  condition;  and  a  change 
in  the  value  of  p is  considered  as  an  anomaly.  With 
amplitude  A  =  22.0  and  Q  =  5.0,  a  sharp  change  in 
the  behavior  is  noticed  around  P  =  0.29,  possibly 
due  to  bifurcation.  The  phase  plots  and  time- 
response  plots,  depicting  this  drastic  change  beha¬ 
vior,  are  not  presented  here  as  they  are  provided  in 
an  earlier  publication  [1]. 

The  objective  of  anomaly  detection  is  to  identify 
small  changes  in  the  parameter  ft  as  early  as  possible 
and  well  before  it  manifests  a  drastic  change  in  the 
system  dynamics.  The  details  of  the  experimental 
apparatus  are  provided  in  [22].  Time  series  data  of 
the  signal  y(t)  from  the  experimental  apparatus  is 
used  for  symbolic  analysis. 

The  first  step  in  the  analysis  is  selection  of  the 
wavelet  basis.  The  time  series  data  of  the  signal  and 
a  scaled  and  translated  version  of  the  wavelet 
‘gausl’  are  shown  in  the  left-hand  plate  of  Fig.  7. 
For  the  purpose  of  comparison,  the  right-hand  plate 
of  Fig.  7  shows  the  same  time  series  data  of  the 
signal  and  a  scaled  and  translated  version  of  the 
wavelet  ‘dbl’  that  was  used  in  [1]  for  wavelet 
analysis.  Since  ‘gausl’  matches  the  shape  of  the 
signal  more  closely  than  ‘dbl’,  the  wavelet  ‘gausl’  is 

Signal(Dash-dot)  -  gausl  (Solid)  Signal(Dash-dot)  -  dbl  (Solid) 


Fig.  7.  Profiles  of  signal  and  wavelet. 
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better  suited  for  STSA  of  this  data  sequence  than 
‘dbl\  Once  the  wavelet  is  chosen,  the  next  step  is 
identification  of  the  frequencies  of  interest.  The  top 
and  bottom  plates  of  Fig.  8  depict  the  power  spectra 
of  the  system  for  /?  —  0.1  and  0.29,  respectively. 

Fig.  8  shows  that  frequencies  in  the  neighborhood 
of  0.54  FIz,  though  present  at  the  nominal  condition 
/?  =  0.10,  are  absent  at  /?  =  0.29.  The  wavelet 
coefficients  at  scales,  corresponding  to  the  pseudo¬ 
frequency  of  0.54  Hz,  would  be  smaller  in  magni¬ 
tude  in  the  anomalous  condition(s)  when  compared 
with  those  in  the  nominal  condition.  Hence,  by 
choosing  scales  that  correspond  to  pseudo-frequen¬ 
cies  around  0.54  Hz,  early  detection  can  be  achieved 
more  effectively. 

The  next  aspect  of  anomaly  detection  via  STSA  is 
the  choice  of  number  of  symbols,  i.e.,  cardinality  |T| 
of  the  symbol  alphabet  E.  The  scale  series  data,  at 
the  nominal  condition,  is  partitioned  into  a  symbol 
sequence  starting  with  |T|  =  2  and  the  threshold 
parameter  e/,  is  chosen  to  be  0.2.  Fig.  9  depicts  the 
change  in  entropy  h  versus  the  number  of  symbols 
\E\ .  It  is  seen  that  h  monotonically  decreases  with 
IZ)  and  becomes  less  than  £/,  when  |Z|  =  8. 
Accordingly,  the  number  of  symbols  |Z|  is  chosen 
to  be  eight.  A  smaller  value  of  £/,  results  in  increased 
number  of  symbols,  which  would  increase  computa¬ 
tion  with  (possibly)  no  significant  gain  in  accuracy 
of  anomaly  detection. 

The  partition  is  obtained  using  data  obtained 
under  the  nominal  (/?  =  0.1)  condition.  Once  the 
partition  is  generated,  it  remains  invariant.  As  the 
dynamical  behavior  of  the  system  changes  due  to 
variations  in  /?,  the  statistical  characteristics  of  the 


p  =  o.i 
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Fig.  8.  Power  spectrum  plots  at  nominal  and  anomalous 
conditions. 


Fig.  9.  Selection  of  number  of  symbols  from  change  in  entropy. 


symbol  sequences  are  also  altered  and  so  do  the 
symbol  probabilities.  A  measure  could  be  induced 
on  the  symbol  probability  vectors  obtained  under 
different  anomalous  conditions,  to  quantify  these 
changes.  Such  a  measure  is  called  the  anomaly 
measure  M.  The  metric  Mk  =  r/(p0,pk)  is  an 
anomaly  measure,  where  p0  and  pk  represent  the 
symbol  probability  vectors  under  nominal  and 
anomalous  conditions,  respectively.  A  candidate 
anomaly  measure  is  the  angle  between  the  symbol 
probability  vectors  under  nominal  and  anomalous 
conditions.  This  measure  is  defined  as 

Mk  =  arccosf — ^°’  ^ — V  (14) 

V II  Po  lln  II  Pk  II 2/ 

where  (x,y)  is  the  inner  product  of  the  vectors  x  and 
y;  and  ||x|]2  is  the  Euclidean  norm  of  x. 

Fig.  10  compares  the  profiles  of  the  anomaly 
measures  Mk  in  Eq.  (14)  obtained  with  wavelet 
‘gausl’  under  ME  partitioning  and  uniform  parti¬ 
tioning.  With  increasing  from  0.1,  there  is  a 
gradual  increase  in  the  anomaly  measure  much 
before  the  abrupt  change  in  the  vicinity  of  /i  =  0.29 
takes  place.  This  indicates  growth  of  the  anomaly 
even  before  any  notable  change  in  the  dynamical 
behavior  takes  place.  At  this  point,  the  anomaly 
measure  starts  increasing  relatively  more  rapidly 
suggesting  the  onset  of  a  forthcoming  catastrophic 
failure.  Under  ME  partitioning,  the  larger  values  of 
the  anomaly  measure  at  smaller  values  of  /!  and 
gradual  increase  in  both  slope  and  curvature  of  the 
anomaly  measure  curve  would  facilitate  anomaly 
detection  significantly  before  it  is  possible  to  do  so 
under  uniform  partitioning.  Therefore,  with  regard 
to  early  detection  of  anomalies,  ME  partitioning 
appears  to  be  more  effective  than  uniform  parti¬ 
tioning. 


V.  Rajagopalan,  A.  Ray  /  Signal  Processing  86  (2006)  3309-3320 


3317 


Fig.  10.  Maximum  entropy  partitioning  and  uniform  partition¬ 
ing. 


Fig.  1 1.  Anomaly  detection  on  the  electronic  system  apparatus. 


Fig.  1 1  depicts  the  anomaly  measure  plots 
obtained  with  wavelet  partitioning  (‘gausl’)  and 
phase-space  partitioning  (SFNN).  These  two  pro¬ 
files  of  anomaly  measure  are  generated  from  the 
same  time  series  data,  where  the  number  of  symbols 
|T|  =  8  in  both  cases.  It  is  observed  that  ME 
partitioning  with  ‘gausl’  wavelet  is  comparable  to 
SFNN  partitioning.  However,  in  this  problem,  the 
execution  time  for  SFNN  to  generate  the  partition  is 
found  to  be  Ri  4  h,  while  that  for  ME  partitioning  is 
Rt  100  ms  on  the  same  computer.  Therefore,  it  may 
be  inferred  from  this  experiment  that  ME  partition¬ 
ing  is  computationally  several  orders  of  magnitude 
less  intensive  than  SFNN  partitioning  while  they 
yield  similar  performance  from  the  perspectives  of 
anomaly  detection. 


5.2.  Structural  damage  detection  in  mechanical 
systems 

This  example  demonstrates  efficacy  of  the  STSA 
method  for  early  detection  of  damage  in  mechanical 
structures.  Laboratory  experiments  have  been  con¬ 
ducted  on  a  multi-degree-of-freedom  mechanical 
vibration  apparatus  [3],  The  mechanical  system  in 
the  apparatus  is  persistently  excited  at  a  frequency 
of  10.4  Hz,  which  is  a  close  approximation  of  one  of 
resonance  frequencies  of  the  mechanical  structure. 
During  the  experiments,  time  series  data  have  been 
collected  from  the  measurements  of  displacement 
sensors,  and  each  set  contains  30  s  of  data  under 
persistent  vibratory  motion  of  the  mass-beam 
system.  The  resulting  cyclic  stresses  induce  (irrever¬ 
sible)  fatigue  crack  damage  in  the  critical  structures, 
which  cause  gradual  reduction  in  stiffness.  Conse¬ 
quently,  the  statistics  of  time  series  data  undergo 
changes.  The  objective  here  is  to  detect  these 
changes  as  early  as  possible  in  real  time. 

The  first  data  set,  which  is  dominated  by  a 
sinusoid  of  frequency  ~  10.4  Hz  and  represents  the 
nominal  behavior  of  the  mechanical  vibration 
system,  is  considered  to  be  the  reference  point. 
The  wavelet  ‘gaus9’  is  chosen  for  analysis  because 
this  wavelet  closely  matches  the  shape  of  the  signal. 
A  small  set  of  scales  around  the  frequency  of 
10.4  Hz  are  chosen  as  per  procedure  outlined  in 
Section  2.  The  alphabet  size  is  chosen,  based  on  the 
entropy  rate,  as  eight,  i.e.,  |T|  =  8. 

Fig.  12  depicts  two  plots  of  anomaly  measure, 
which  are  obtained  from  the  same  data  set  with 
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Fig.  12.  Anomaly  detection  on  the  mechanical  vibration  appa¬ 
ratus. 
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different  partitioning  methods. 

•  SFNN  partitioning  [14]. 

•  ME  partitioning  with  wavelet  ‘gaus9’. 

It  is  observed  in  Fig.  12  that  ME  partitioning  with 
‘gaus9’  wavelet  is  comparable  to  SFNN  partition¬ 
ing.  However,  in  this  problem,  the  execution  time 
for  ME  partitioning  is  about  five  orders  of 
magnitude  less  than  that  for  SENN  partitioning. 
Hence,  it  may  be  inferred  from  this  experiment  that 
ME  partitioning,  is  better  suited  for  real-time 
structural  damage  detection  in  mechanical  systems. 

5.3.  Fatigue  damage  detection  in  polycrystalline 
alloys 

This  example  presents  a  comparison  between 
wavelet  space  and  phase-space  partitioning  in  the 
context  of  fatigue  crack  damage  detection  in 
polycrystalline  alloys.  The  details  of  the  experimen¬ 
tal  apparatus,  equipped  with  an  ultrasonic  flaw 
detector  and  an  optical  microscope,  on  which 
experiments  were  conducted  are  reported  in  [5] 
and  references  therein.  Low-cycle  fatigue  tests  have 
been  conducted  with  specimens,  made  of  aluminum 
alloy  7075-T6.  A  sinusoidal  load  with  maximum 
and  minimum  loads  of  87  and  4.85  MPa,  respec¬ 
tively,  is  applied.  A  significant  amount  of  internal 
damage  occurs  before  the  crack  appears  on  the 
surface  of  the  specimen,  where  it  is  observed  by  the 
optical  microscope.  This  internal  damage  caused  by 
multiple  small  cracks  and  dislocations  affect  the 
ultrasonic  waves  as  they  pass  through  the  regions 
where  these  faults  have  developed.  This  phenomen¬ 
on  causes  signal  distortion  and  attenuation  at  the 
receiver  end  of  the  ultrasonic  flaw  detector. 

The  crack  propagation  stage  starts  when  the 
internal  damage  eventually  develops  into  a  single 
large  crack.  Subsequently,  the  crack  growth  rate 
increases  rapidly  and  when  the  crack  is  sufficiently 
large,  the  transmitted  ultrasonic  signal  is  almost 
completely  attenuated.  The  process  of  obtaining  the 
ultrasonic  time  series  data  is  described  in  [5].  The 
results  of  S77S74-based  anomaly/damage  detection, 
obtained  with  ME  and  SFNN  partitioning  methods, 
are  presented  below. 

Wavelet  ‘gaus2’  is  used  for  obtaining  the  coeffi¬ 
cients  that  are  stacked  to  form  the  scale  series  data 
set.  The  alphabet  size  T  is  chosen  to  be  eight.  The 
scale  series  data  set  is  partitioned  by  imposing  the 
ME  criterion  to  generate  the  symbols.  Fig.  13 


Fig.  13.  Anomaly  measure  plots  for  fatigue  crack  detection. 


depicts  two  anomaly  measure  plots  that  are 
obtained  from  the  same  data  set  by  using  ME  and 
SFNN  partitioning. 

While  the  SFNN  partitioning  yields  slightly  higher 
values  of  the  anomaly  measure  and  comparable 
slope,  the  profile  of  wavelet  partitioning  is  more 
smooth  and  yields  a  larger  change  in  the  curvature 
around  32  kilocycles,  which  is  an  early  warning  for 
the  forthcoming  failure.  Simultaneous  consideration 
of  the  anomaly  measure,  slope,  and  curvature 
provides  a  robust  method  of  failure  prediction  and 
reduces  the  probability  of  false  alarms. 

Similar  to  the  previous  experiments,  it  is  observed 
that  the  execution  time  for  ME  partitioning  is 
approximately  five  orders  of  magnitude  less  than 
that  for  SFNN  partitioning.  Hence,  it  may  be 
inferred  that  ME  partitioning,  is  better  suited  for 
real-time  fatigue  damage  detection  in  polycrystalline 
alloys. 

6.  Summary  and  conclusions 

This  paper  presents  a  novel  method  of  partition¬ 
ing  the  data  space  for  symbolic  time  series  analysis 
(STSA).  In  this  approach,  symbols  are  generated 
from  the  wavelet  coefficients  of  the  time  series  data, 
instead  of  obtaining  them  directly  from  the  time 
series  data.  Various  aspects  of  this  method,  such  as 
selection  of  wavelet  basis  and  scale  range,  are 
systematically  investigated  for:  (i)  suppression  of 
noise  and  spurious  disturbances  and  (ii)  enhance¬ 
ment  of  sensitivity  to  changes  in  signal  dynamics. 
The  advantages  of  using  wavelet-based  partitioning 
over  phase-space  partitioning  are  demonstrated  by 
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numerical  simulation  and  laboratory  experimenta¬ 
tion.  It  has  been  shown  that  measurement  noise 
suppression  results  in  robust  symbol  generation. 
The  symbol  sequences,  generated  from  the  wavelet 
coefficients  of  a  noisy  signal,  are  able  to  capture  the 
changes  in  signal  information  better  than  those 
obtained  directly  from  the  time  series  data  of  the 
signal.  It  is  also  shown  that  the  choice  of  an 
appropriate  wavelet  and  scales  significantly  im¬ 
proves  computational  efficiency  and  thereby  en¬ 
hances  implementation  of  the  anomaly  detection 
technique  for  real-time  applications.  An  entropy- 
based  algorithm  is  introduced  for  selection  of  the 
symbol  alphabet  size. 

A  partitioning  method,  based  on  maximum 
entropy,  is  introduced  and  is  compared  with  the 
uniform  partitioning  method.  The  performance  of 
wavelet-based  partitioning  has  been  tested  via  experi¬ 
mentation  for:  (i)  anomaly  detection  on  a  nonlinear 
electronic  system  apparatus  [1];  (ii)  structural  damage 
detection  on  a  mechanical  vibration  system  apparatus 
[3];  and  (iii)  damage  detection  in  polycrystalline  alloys 
on  a  Fatigue  Testing  apparatus  [5], 

Wavelet-based  maximum  entropy  (ME)  parti¬ 
tioning  has  been  compared  with  symbolic  false 
nearest  neighbor  ( SFNN)  partitioning  [14]  with 
regard  to  anomaly  detection.  Based  on  the  time 
series  data  from  three  experimental  apparatuses,  it 
is  observed  that  the  aforementioned  partitioning 
methods  yield  comparable  results  while  the  compu¬ 
tation  time  for  wavelet-based  ME  partitioning  is 
observed  to  be  several  orders  of  magnitude  smaller 
than  that  for  SFNN  partitioning. 

A  major  conclusion  based  on  this  investigation  is 
that  wavelet-based  ME  partitioning,  combined  with 
an  appropriate  choice  of  wavelet  and  scales, 
significantly  enhances  computational  efficiency  and 
anomaly  detection  capabilities  beyond  what  has 
been  reported  in  literature.  The  field  of  STSA  is 
relatively  new  and  its  application  to  anomaly 
detection  is  very  recent.  Therefore,  the  proposed 
method  of  symbol  generation  for  anomaly  detection 
requires  continued  theoretical  and  experimental 
research.  In  this  context,  future  research  is  recom¬ 
mended  in  the  following  areas: 

•  Exploration  of  lifting  techniques  [23]  for  wavelet 
customization; 

•  Extension  of  ME  partitioning  to  multi-dimen¬ 
sional  time  series; 

•  Noise  reduction  in  time  series  for  robust  anom¬ 
aly/damage  detection. 
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Abstract 

The  paper  presents  an  analytical  tool  for  early  detection  and  online  monitoring  of  fatigue  damage  in  polycrystalline  alloys  that  are 
commonly  used  in  mechanical  structures  of  human-engineered  complex  systems.  Real-time  fatigue  damage  monitoring  algorithms  rely 
on  time  series  analysis  of  ultrasonic  signals  that  are  sensitive  to  micro-structural  changes  occurring  inside  the  material  during  the  early 
stages  of  fatigue  damage;  the  core  concept  of  signal  analysis  is  built  upon  the  principles  of  Symbolic  Dynamics,  Statistical  Pattern  Rec¬ 
ognition  and  Information  Theory.  The  analytical  tool  of  statistical  pattern  analysis  has  been  experimentally  validated  on  a  special-purpose 
test  apparatus  that  is  equipped  with  ultrasonic  flaw  detection  sensors  and  a  travelling  optical  microscope.  The  paper  reports  fatigue  dam¬ 
age  monitoring  of  7075-T6  alloy  specimens,  where  the  experiments  have  been  conducted  under  load-controlled  constant  amplitude  sinu¬ 
soidal  loadings  for  low-cycle  and  high-cycle  fatigue. 

©  2006  Elsevier  Ltd.  All  rights  reserved. 
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1.  Introduction 

Prediction  of  structural  damage  and  quantification  of 
structural  integrity  are  critical  for  safe  and  reliable  opera¬ 
tion  of  human-engineered  complex  systems.  Fatigue  dam¬ 
age  is  one  of  the  most  commonly  encountered  sources  of 
structural  degradation  during  both  nominal  and  off-nomi¬ 
nal  operations  of  such  systems  [1].  Therefore,  it  is  necessary 
to  develop  diagnosis  and  prognosis  capabilities  for  reliable 
and  safe  operation  of  the  system  and  for  enhanced  avail¬ 
ability  of  its  service  life.  Many  model-based  techniques 
have  been  reported  in  literature  for  structural  health  mon¬ 
itoring  and  life  prediction  [2-5].  Apparently,  no  existing 
model,  solely  based  on  the  fundamental  principles  of 
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molecular  physics  [6],  can  adequately  capture  the  dynami¬ 
cal  behavior  of  fatigue  damage  at  the  grain  level.  In  gen¬ 
eral,  model-based  approaches  are  critically  dependent  on 
initial  defects  in  the  material  micro-structure,  which  may 
randomly  form  crack  nucleation  sites  and  are  difficult  to 
model  [1,7].  Small  deviations  in  initial  conditions  and  crit¬ 
ical  parameters  may  produce  large  bifurcations  in  the 
expected  dynamical  behavior  of  fatigue  damage  [8],  In 
addition,  fluctuations  in  usage  patterns  (e.g.,  random  over¬ 
loads)  and  environmental  conditions  (e.g.,  temperature  and 
humidity)  may  adversely  affect  the  service  life  of  mechani¬ 
cal  systems.  As  such,  fatigue  damage  is  described  as  a  sto¬ 
chastic  phenomenon  that  emphasizes  the  need  for  online 
monitoring  using  sensing  devices  which  can  provide  useful 
and  reliable  estimates  of  the  anomalies  at  an  early  stage  [9]. 

Information-based  fatigue  damage  monitoring  using  dif¬ 
ferent  sensors  (e.g.,  acoustic  emission,  eddy  currents  and 
ultrasonic)  has  been  proposed  in  recent  literature  [10-12]. 
The  capabilities  of  electrochemical  sensors  [13]  and  thermal 
imaging  techniques  [14]  have  also  been  investigated  for 
structural  failure  analysis.  The  traditional  analysis  methods 
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Nomenclature 

D 

window  length  on  a  symbolic  sequence 

( 7 

a  symbol  on  a  symbolic  sequence 

n 

number  of  states  of  the  finite  state  machine 

■*!k 

transition  probability  from  state  tf  to  qk 

vk 

state  probability  vector  at  time  epoch  tk 

nk 

state  transition  matrix  at  time  epoch  tk 

p(») 

Probability  of  the  event  • 

V 

anomaly  measure  computed  at  time  epoch  tk 

9l 

/'th  state  of  the  finite  state  machine 

/th  block  of  the  partition  of  the  phase  space 

Q 

set  of  all  states  of  the  finite  state  machine 

Q 

phase  space  of  a  dynamical  system 

s 

dynamical  systems  entropy  of  the  symbol 

I 

alphabet  set 

tk 

sequence 
slow  time  epoch 

1*1 

size  of  the  alphabet  set 

using  acoustic  emission  technique  are  used  to  correlate  the 
signal  parameters  (such  as  the  acoustic  emission  counts,  the 
peak  amplitudes  and  the  energy)  with  the  defect  formation 
mechanisms  and  to  provide  a  quantified  estimate  of  faults. 
Acoustic  emission  technique  has  been  investigated  by  sev¬ 
eral  researchers  for  its  sensitivity  to  the  activities  occurring 
inside  the  material  micro-structure  for  early  detection  of 
fatigue  and  fracture  failures  [15-19].  However,  the  major 
drawback  of  acoustic  emission  technique  is  poor  perfor¬ 
mance  in  noisy  environments  where  signal-noise  separa¬ 
tion  becomes  a  difficult  task. 

The  eddy  current  technique  is  based  on  the  principles  of 
electromagnetic  induction.  When  a  source  of  alternating 
current  is  supplied  to  a  conductor,  a  magnetic  field  devel¬ 
ops,  which  induces  eddy  currents  in  the  material.  The  pres¬ 
ence  of  faults  in  the  material  affect  the  eddy  current  flow 
patterns,  which  can  be  detected  for  prediction  and  estima¬ 
tion  of  the  structural  damage  [20-22].  The  advantages  of 
eddy  current  inspection  technique  include  sensitivity  to 
small  cracks  and  other  defects,  portability  of  sensor  equip¬ 
ment,  minimum  part  preparation,  and  non-contact  evalua¬ 
tion.  However,  the  limitations  of  the  eddy  current 
inspection  technique  are  the  depth  of  penetration  and  it 
can  be  used  to  detect  only  surface  and  near  surface  defects. 
Furthermore,  only  conductive  materials  can  be  inspected. 

Ultrasonic  sensors  function  by  emitting  high-frequency 
ultrasonic  pulses  that  travel  through  the  specimen  and 
are  received  by  the  transducers  at  the  other  end.  As  with 
the  propagation  of  any  wave,  it  is  possible  that  disconti¬ 
nuities  in  the  propagation  media  will  cause  additive  and 
destructive  interference.  Since  material  characteristics 
(e.g.,  voids,  dislocations  and  short  cracks)  influence  the 
ultrasonic  impedance,  a  small  fault  in  the  specimen  is 
likely  to  change  the  signature  of  the  signal  at  the  receiver 
end  [23,24,9,25,26].  Therefore,  the  ultrasonic  signals  can 
be  used  to  capture  some  of  the  minute  details  and  small 
changes  during  the  early  stages  of  fatigue  damage,  which 
may  not  be  possible  to  detect  by  an  optical  microscope 
[9].  Moreover,  ultrasonic  sensing  is  applicable  to  real¬ 
time  applications  and  the  sensing  probes  can  be  easily 
installed  at  the  critical  sites.  Ultrasonic  sensing  is  also 
robust  to  noisy  environments  since  the  externally  excited 
waves  are  of  very  high  frequency  and  they  do  not  inter¬ 


fere  with  small  disturbances.  As  such,  this  paper  explores 
the  ultrasonic  sensing  technique  to  examine  small  micro- 
structural  changes  during  early  stages  of  fatigue  damage 
evolution. 

The  above  discussions  evince  the  fact  that  time  series 
analysis  of  sensor  data  is  essential  for  real-time  detection 
and  monitoring  of  fatigue  damage.  From  this  perspective, 
the  paper  presents  symbolic  time  series  analysis  (STSA) 
[27]  of  ultrasonic  sensor  signals  for  early  detection  of  evolv¬ 
ing  anomalies.  The  STSA  method  is  an  information-theo¬ 
retic  pattern  identification  tool  that  is  built  upon  a 
fixed-structure,  fixed-order  Markov  chain  [28];  it  has  been 
validated  by  comparison  with  existing  pattern  recognition 
techniques  such  as  Principal  Component  Analysis  (PCA) 
and  Artificial  Neural  Networks  (ANN)  [29,30].  A  com¬ 
puter-controlled  special  purpose  test  apparatus,  equipped 
with  multiple  sensing  devices  (e.g.,  ultrasonics  and  optical 
microscope)  for  damage  analysis,  has  been  used  to  experi¬ 
mentally  validate  the  STSA  method  of  fatigue  damage 
detection.  Experiments  have  been  conducted  under  differ¬ 
ent  loading  conditions  on  7075-T6  aluminum  alloy 
specimens. 

The  paper  is  organized  in  seven  sections  including  the 
present  section.  Section  2  presents  an  overview  of  anomaly 
detection  methodology  using  a  two-time-scale  approach. 
Section  3  describes  the  underlying  concepts  and  essential 
features  of  symbolic  time  series  analysis  for  anomaly  detec¬ 
tion.  Section  4  provides  a  brief  description  of  the  fatigue 
damage  test  apparatus.  Section  5  presents  the  experimental 
procedure  and  the  application  of  STSA  for  fatigue  damage 
monitoring.  Section  6  presents  the  results  and  discussion 
on  early  detection  of  fatigue  damage  under  different  load¬ 
ing  conditions.  The  paper  is  concluded  in  Section  7  along 
with  recommendations  for  future  research. 

2.  Methodology  for  fatigue  damage  monitoring 

Traditional  tools  like  S-N  curves  and  Goodman  dia¬ 
grams  are  not  effective  for  real-time  damage  prediction. 
This  problem  is  circumvented  by  information-based  diag¬ 
nosis  and  prognosis  tools  that  are  capable  of  identifying 
damage  patterns  in  real  time  from  the  statistical  behavior 
of  sensor  (e.g.,  ultrasonic)  data  sequences.  In  this  context, 
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Fig.  1.  Pictorial  view  of  the  two  time  scales:  (1)  slow  time  scale  where  anomalies  evolve  and  (2)  fast  time  scale  where  data  acquisition  is  done. 


fatigue  damage  monitoring  is  formulated  as  a  two-time- 
scale  problem.  The  sampling  frequency  for  data  acquisition 
is  required  to  be  several  orders  of  magnitude  faster  than  the 
time  period  of  damage  evolution.  Fatigue  damage  monitor¬ 
ing  is  a  two-time-scale  problem  as  delineated  below: 

•  The  fast  time  scale  is  related  to  the  response  time  of 
machinery  operation.  Over  the  span  of  a  given  time  ser¬ 
ies  data  sequence,  the  structural  dynamic  behavioral  sta¬ 
tistics  of  the  system  are  assumed  to  remain  invariant, 
i.e.,  the  process  is  assumed  to  have  statistically  station¬ 
ary  dynamics  at  the  fast  time  scale. 

•  The  slow  time  scale  is  related  to  the  time  span  over  which 
the  process  may  exhibit  non-stationary  dynamics  due  to 
(possible)  evolution  of  anomalies.  Thus,  an  observable 
non-stationary  behavior  can  be  associated  with  anoma¬ 
lies  evolving  at  a  slow  time  scale. 

A  pictorial  view  of  the  two  time  scales  is  presented  in 
Fig.  1.  In  general,  a  long  time  span  in  the  fast  time  scale 
is  a  tiny  (i.e.,  several  orders  of  magnitude  smaller)  interval 
in  the  slow  time  scale.  For  example,  evolution  of  fatigue 
damage  in  structural  materials  (causing  a  detectable  change 
in  the  dynamics  of  the  system)  occurs  on  the  slow  time  scale 
(possibly  in  the  order  of  months  or  years);  fatigue  damage 
behavior  is  essentially  invariant  on  the  fast  time  scale 
(approximately  in  the  order  of  seconds  or  minutes).  Never¬ 
theless,  the  notion  of  fast  and  slow  time  scales  is  dependent 
on  the  specific  application,  loading  conditions  and  operat¬ 
ing  environment.  As  such,  from  the  perspective  of  fatigue 
monitoring,  sensor  data  acquisition  is  done  on  the  fast  time 
scale  at  different  slow  time  epochs  separated  by  fixed  inter¬ 
vals  on  the  slow  time  scale.  Specifically,  the  objective  of  this 
paper  is  to  demonstrate  real-time  monitoring  of  fatigue 
damage  by  detecting  small  changes  in  the  statistical  pat¬ 
terns  of  the  ultrasonic  sequences  using  the  STSA  approach. 

3.  Symbolic  time  series  analysis  for  anomaly  detection 

This  section  presents  the  underlying  concepts  and  essen¬ 
tial  features  of  symbolic  time  series  analysis  [27]  for  anom¬ 
aly  detection  in  complex  dynamical  systems  [28].  While  the 
details  are  reported  in  previous  publications  [28,29,31];  a 
brief  review  of  this  method  is  presented  here  for  complete¬ 
ness  and  clarity  of  the  paper.  The  principle  of  symbolic 
dynamics  is  based  on  the  transformation  of  a  data  sequence 
(e.g.,  time  series  data)  to  a  symbol  sequence  by  partitioning 


a  compact  region  Q  of  the  phase  space,  over  which  the  tra¬ 
jectory  evolves,  into  finitely  many  discrete  blocks  as  shown 
in  Fig.  2.  Let  {</>,,<f2, . .  be  a  partitioning  of  Q,  such 
that  it  is  exhaustive  and  mutually  exclusive  set,  i.e., 

m 

|J  <l>j  =  Q  and  <P,  f)  =  0  V/  f  k.  (1) 

y=i 

Each  block  'If  is  labelled  as  the  symbol  OjcD,  where  the 
symbol  set  S  is  called  the  alphabet  set  consisting  of  in  dif¬ 
ferent  symbols  (i.e.,  m  =  |T|).  As  the  system  evolves  in  time, 
it  travels  through  various  blocks  in  its  phase  space  and  the 
corresponding  symbol  a yeT  is  assigned  to  it,  thus  convert¬ 
ing  a  data  sequence  to  a  symbol  sequence  . . .  ah  ah  . . .  ait . . . 
[32,33].  Fig.  2  exemplifies  the  partitioning  of  the  phase 
space  where  each  block  is  assigned  a  particular  symbol 
such  that  a  symbol  sequence  is  generated  from  the  phase 
space  at  a  given  slow  time  epoch.  Once  the  symbol 
sequence  is  obtained,  the  next  step  is  construction  of  a 
finite  state  machine  [34],  These  steps  are  explained  in 
details  in  the  following  subsections. 

3.1.  Wavelet  space  partitioning 

A  crucial  step  in  symbolic  time  series  analysis  is  parti¬ 
tioning  of  the  phase  space  for  symbol  sequence  generation 
[27],  Several  partitioning  techniques  have  been  reported  in 
literature  for  symbol  generation  [35-37],  primarily  based 
on  symbolic  false  neighbors.  These  techniques  rely  on  par¬ 
titioning  the  phase  space  and  may  become  cumbersome 
and  extremely  computation-intensive  if  the  dimension  of 
the  phase  space  is  large.  Moreover,  if  the  time  series  data 
is  noise-corrupted,  then  the  symbolic  false  neighbors  would 
rapidly  grow  in  number  and  require  a  large  symbol  alpha¬ 
bet  to  capture  the  pertinent  information  on  the  system 
dynamics.  Therefore,  symbolic  sequences  as  representa¬ 
tions  of  the  system  dynamics  should  be  generated  by  alter¬ 
native  methods  because  phase-space  partitioning  might 
prove  to  be  a  difficult  task  in  the  case  of  high-dimensions 
and  presence  of  noise.  The  wavelet  transform  [38]  largely 
alleviates  these  shortcomings  and  is  particularly  effective 
with  noisy  data  from  high-dimensional  dynamical  systems. 

This  paper  has  adopted  a  wavelet-based  partitioning 
approach  [28,31]  for  construction  of  symbol  sequences 
from  the  time  series  data.  In  this  method,  the  time  series 
data  are  first  converted  by  wavelet  transform,  where  wave¬ 
let  coefficients  are  generated  at  different  scales  and  time 
shifts.  The  graphs  of  wavelet  coefficients  versus  scale,  at 
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Fig.  2.  STSA-based  pattern  identification. 
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selected  time  shifts,  are  stacked  starting  with  the  smallest 
value  of  scale  and  ending  with  its  largest  value  and  then 
back  from  the  largest  value  to  the  smallest  value  of  the 
scale  at  the  next  instant  of  time  shift.  The  arrangement  of 
the  resulting  scale  series  data  in  the  wavelet  space  is  similar 
to  that  of  the  time  series  data  in  the  phase  space.  The  wave¬ 
let  space  is  partitioned  with  alphabet  size  |T|  into  segments 
of  coefficients  on  the  ordinate  separated  by  horizontal  lines 
such  that  the  regions  with  more  information  are  parti¬ 
tioned  finer  and  those  with  sparse  information  are  parti¬ 
tioned  coarser.  This  is  achieved  by  maximizing  the 
Shannon  entropy  [39],  which  is  defined  as 

hi 

s=  -^PAogiPi),  (2) 

1=1 

where  p,  is  the  probability  of  the  ith  state  and  summation  is 
taken  over  all  possible  states.  Uniform  probability  distribu¬ 
tion  of  states  is  a  consequence  of  maximum  entropy  that 
makes  the  partition  coarser  in  regions  of  low-data  density 
and  finer  in  regions  of  high-data  density. 

3.2.  State  machine  construction 

The  partitioning  as  described  in  the  previous  section  is 
performed  at  time  epoch  t0  of  the  nominal  condition  that 
is  chosen  to  be  a  healthy  condition  having  zero  anomaly 
measure.  A  finite  state  machine  [34]  is  then  constructed, 
where  the  states  of  the  machine  are  defined  corresponding 
to  a  given  alphabet  X  and  window  length  D.  The  alphabet 
size  T|  is  the  total  number  of  partitions  while  the  window 
length  D  is  the  length  of  consecutive  symbol  words  forming 
the  states  of  the  machine  [28].  The  states  of  the  machine  are 


chosen  as  all  possible  words  of  length  D  from  the  symbol 
sequence,  thereby  making  the  number  n  of  states  to  be 
equal  to  the  total  permutations  of  the  alphabet  symbols 
within  words  of  length  D,  (i.e.,  n  f  \X\D;  some  of  which 
may  be  forbidden  with  zero  probability  of  occurrence). 
For  example,  if  X  =  {0, 1},  i.e.,  |27|  =  2  and  D  =  2,  then 
the  number  of  states  is  n  f  \X\D  =  4;  and  the  possible  states 
are  Q  =  {00,01,10,11). 

The  choice  of  |T|  and  D  depends  on  specific  experiments, 
noise  level  and  also  the  available  computation  power.  A 
large  alphabet  may  be  noise-sensitive  while  a  small  alphabet 
could  miss  the  details  of  signal  dynamics.  Similarly,  a  high- 
value  of  D  is  extremely  sensitive  to  small  signal  distortions 
but  would  lead  to  a  large  number  of  states  requiring  more 
computation  power.  Using  the  symbol  sequence  generated 
from  the  time  series  data,  the  state  machine  is  constructed 
on  the  principle  of  sliding  block  codes  [32]  as  explained 
below.  The  window  of  length  D  on  the  symbol  sequence 
. . .  cy  oh  . . .  aik ...  is  shifted  to  the  right  by  one  symbol,  such 
that  it  retains  the  last  (D  —  1)  symbols  of  the  previous  state 
and  appends  it  with  the  new  symbol  ait  at  the  end.  The 
symbolic  permutation  in  the  current  window  gives  rise  to 
a  new  state.  The  machine  constructed  in  this  fashion  is 
called  /)- Markov  machine  [28]  because  of  its  Markov 
properties. 

Definition  3.1.  A  symbolic  stationary  process  is  called 
/)- Markov  if  the  probability  of  the  next  symbol  depends 
only  on  the  previous  D  symbols,  i.e., 

p{oiJcu  ■  ■  ■  ■■■)=  p{°klaU  ■  ■  ■ 

The  finite  state  machine  constructed  above  has  /t- Mar¬ 
kov  properties  because  the  probability  of  occurrence  of 
symbol  oit  on  a  particular  state  depends  only  on  the 
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configuration  of  that  state,  i.e.,  the  previous  D  symbols. 
Once  the  partitioning  alphabet  S  and  word  length  D  are 
determined  at  the  nominal  condition  (time  epoch  t0),  they 
are  kept  constant  for  all  (slow  time)  epochs  {t\,  t2,  ■  ■ .,  tk. . .}, 
i.e.,  the  structure  of  the  machine  is  fixed  at  the  nominal 
condition.  That  is,  the  partitioning  and  the  state  machine 
structure  generated  at  the  nominal  condition  serve  as  the 
reference  frame  for  data  analysis  at  subsequent  time 
epochs.  For  D  =  1,  the  set  of  states  bears  an  equivalence 
relation  to  the  alphabet  Z  of  symbols  [40].  The  states  of 
the  machine  are  marked  with  the  corresponding  symbolic 
word  permutation  and  the  edges  joining  the  states  indicate 
the  occurrence  of  an  event  air  The  occurrence  of  an  event 
at  a  state  may  keep  the  machine  in  the  same  state  or  move 
it  to  a  new  state.  The  language  of  the  machine  is  usually 
incomplete  in  the  sense  that  all  states  might  not  be  reach¬ 
able  from  a  given  state. 

Definition  3.2.  The  probability  of  transitions  from  state  qj 
to  state  qk  belonging  to  the  set  Q  of  states  under  a 
transition  6:  Q  x  Z  — >  Q  is  defined  as 

njk  =  P{o  G  Z\ S(qp  a)  -►  qk)\  ^  njk  =1.  (3) 

k 

Thus,  for  a  77-Markov  machine,  the  irreducible  stochastic 
matrix  77  =  [7i,y]  describes  all  transition  probabilities  between 
states  such  that  it  has  at  most  |27|jD+i  nonzero  entries.  The  left 
eigenvector  p  corresponding  to  the  unit  eigenvalue  of  77  is  the 
state  probability  vector  under  the  (fast  time  scale)  stationary 
condition  of  the  dynamical  system  [28].  On  a  given  symbol 
sequence  . . .  er„  ah  . .  ,ah. . .  generated  from  the  time  series 
data  collected  at  slow  time  epoch  tk,  a  window  of  length 
(7))  is  moved  by  keeping  a  count  of  occurrences  of  word 
sequences  ah  . . .  aiDaiD+i  and  ah  . . .  aiD  which  are  respectively 
denoted  by  7V(crfl  . . .  aiDaiD+l)  and  N(c r,-,  . . .  aiD).  Note  that  if 
N(ah  . . .  aiD)  =  0,  then  the  state  q  =  au  . . .  oin  e  Q  has  zero 
probability  of  occurrence.  For  N(ah  . . .  ain)  /  0,  the  transi¬ 
tions  probabilities  are  then  obtained  by  these  frequency 
counts  as  follows: 


nik  =  P[qMj] 


p[<ik,  qj\ 

p[%] 


N{ah  . . .  oiDo) 
N((Th  ■  ■  ■  aiD)  ' 


P{ah  . . .  aiDa) 
P(ah  . . .  oiD ) 


Ttjk 


(4) 


where  the  corresponding  states  are  denoted  by 
qj  =  Oi ,  ah  . . .  a iD  and  qk  =  ah  . . .  aiDa.  The  time  series  data 
under  the  nominal  condition  (set  as  a  benchmark)  gener¬ 
ates  the  state  transition  matrix  11°  that,  in  turn,  is  used  to 
obtain  the  state  probability  vector  p°  whose  elements  are 
the  stationary  probabilities  of  the  state  vector,  where  p°  is 
the  left  eigenvector  of  77°  corresponding  to  the  (unique) 
unit  eigenvalue.  Subsequently,  state  probability  vectors 
p',p2, . .  .,pfc, . . .  are  obtained  at  slow  time  epochs 
t\,  ?2, . . .,  tk, .  •  •  based  on  the  respective  time  series  data.  Ma¬ 
chine  structure  and  partitioning  should  be  the  same  at  all 
slow  time  epochs. 


3.3.  Pattern  identification  procedure 

Behavioral  pattern  changes  are  quantified  as  deviations 
from  the  nominal  behavior  (i.e.,  the  probability  distribu¬ 
tion  at  the  nominal  condition).  The  resulting  anomalies 
(i.e.,  deviations  of  the  evolving  patterns  from  the  nominal 
pattern)  are  characterized  by  a  scalar-valued  function, 
called  Anomaly  Measure  t j/  that  is  quasi-static  in  the  fast 
time  scale  and  is  monotonically  non-decreasing  in  the  slow 
time  scale.  The  state  probability  vector  at  any  time  instant 
corresponds  to  a  singleton  point  on  the  unity-radius  hyper¬ 
sphere.  During  fatigue  damage  evolution,  the  tip  of  the 
probability  vector  moves  along  a  path  on  the  surface  of  this 
hypersphere.  The  initial  starting  point  of  the  path  is  the 
probability  vector  with  uniform  distribution  obtained  with 
maximum  entropy  partitioning  (see  Section  3.1).  As  the 
damage  progresses,  the  probability  distribution  changes; 
eventually  when  a  very  large  crack  is  formed,  complete 
attenuation  of  the  ultrasonic  signal  occurs  and  conse¬ 
quently  the  tip  of  the  probability  vector  reaches  a  point 
where  all  states  have  zero  probabilities  of  occurrence  except 
one  which  has  a  probability  one  (i.e.,  a  delta-distribution); 
this  state  corresponds  to  the  partition  region  where  all  data 
points  are  clustered  due  to  complete  attenuation  of  the 
signal. 

In  the  context  of  fatigue  damage,  the  anomaly  measure 
is  formulated  on  the  following  assumptions. 

•  Assumption  #1:  The  damage  evolution  is  an  irreversible 
process  (i.e.,  with  zero  probability  of  self  healing)  and 
implies  the  following  conditions. 

i lik  ^  0;  i lik+e  -  \l/k  Ss  0  W  ^  0  VJfc.  (5) 

•  Assumption  #2:  The  damage  accumulation  between  two 
time  epochs  is  a  path  function,  i.e.,  dependent  on  the 
path  traversed  to  reach  the  target  state  from  the  initial 
state. 

In  the  context  of  fatigue  damage  in  polycrystalline  alloys 
at  room  temperature,  the  crack  length  is  traditionally 
defined  by  a  straight  line  joining  the  starting  point  to  the 
tip  of  the  crack  but,  in  reality,  the  actual  crack  follows  a 
complicated  path  (possibly  fractal  in  ductile  materials).  In 
fact,  at  the  initial  stages  of  fatigue  damage,  there  can  be 
multiple  short  cracks  oriented  in  different  directions. 
Therefore,  crack  length  alone  does  not  provide  complete 
information  on  fatigue  damage  evolution.  Since  ultrasonic 
signals  are  highly  sensitive  to  small  micro-structural 
changes,  signal  distortion  is  a  good  index  of  anomaly 
growth.  The  tip  of  the  probability  vector,  obtained  through 
symbolic  time  series  analysis,  moves  along  a  curved  path 
on  the  surface  of  the  unity-radius  hypersphere  between 
the  initial  point  p°  (i.e.,  uniform  distribution  obtained 
under  maximum  entropy  partitioning)  and  the  final  point 
at  very  large  crack  formation  \>f  (i.e.,  d-distribution  due 
to  complete  attenuation  of  the  signal).  The  phenomenon 
such  as  piling  up  of  dislocations,  strain  hardening  or  reflec- 
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tions  from  multiple  crack  surfaces  affect  the  ultrasonic  sig¬ 
nals  in  a  variety  of  ways.  An  increase  of  the  ultrasonic 
amplitude  is  also  observed  during  very  early  stages  of  fati¬ 
gue  damage  due  to  hardening  of  the  material.  On  the  other 
hand,  ultrasonic  signals  attenuate  sharply  at  the  crack 
propagation  stage  upon  development  of  a  large  crack. 

As  such,  distortion  of  ultrasonic  signals  at  a  single  time 
epoch  may  not  uniquely  determine  the  state  of  fatigue  dam¬ 
age.  The  rationale  is  that  two  signals  may  exhibit  similar 
characteristics  but,  in  terms  of  actual  incurred  damage, 
the  states  are  entirely  different.  Consequently,  fatigue  dam¬ 
age  is  a  path  function  instead  of  being  a  state  function.  This 
assessment  is  consistent  with  assumption  #1  implying  that 
the  damage  evolution  is  irreversible.  That  is,  at  two  differ¬ 
ent  time  epochs,  the  damage  cannot  be  identical  unless  the 
net  damage  increment  is  zero.  Consequently,  by  assump¬ 
tion  #2,  the  anomaly  measure  should  follow  the  traversed 
path  of  the  probability  vector,  not  the  straight  line  joining 
the  end  points  (i.e.,  the  tips  of  the  probability  vectors). 

The  anomaly  measure,  based  on  the  path  between  the 
nominal  state  to  the  completely  damaged  state,  can  be  dif¬ 
ferent  even  for  identical  test  samples  and  under  the  same 
loading  conditions  because  of  the  stochastic  nature  of  fati¬ 
gue  phenomena.  As  such,  analysis  of  a  stochastic  data  set 
collected  under  identical  experimental  conditions  is  essen¬ 
tial  for  identification  of  variations  in  different  data  sets. 
The  following  distance  function  is  derived  between  proba¬ 
bility  vectors  at  two  time  epochs: 

d(p*,p')  =  yV-p'y'V-p')-  (6) 

The  algorithm  for  computation  of  the  anomaly  measure 
i J/  compensates  for  spurious  measurement  and  computation 
noise  in  terms  of  the  sup  norm  which  is  defined  as 
Helloo  =  max(|ei|, . . .,  \em\)  of  the  error  in  the  probability  vec¬ 
tor  (i.e.,  the  maximum  error  in  the  elements  of  the  proba¬ 
bility  vector).  The  algorithm  is  presented  below. 

(i)  i/,0  =  0;  8ijtl  =  0;  p  =  p°;  k  =  1; 

(ii)  if  Up*  -  pll^  >  e  then  8i//k  =  d{ pA,  p)  and  p  «-  pA; 

(iii)  i jjk  =  \jjk-l  +  di/ik; 

(iv)  k  <—  k  +  1;  8il/k  =  0;  go  to  step  (ii). 

The  real  positive  parameter  e,  is  associated  with  the 
robustness  of  the  measure  against  measurement  and  com¬ 
putation  noise  and  is  identified  by  performing  an  experi¬ 
ment  with  a  sample  with  no  notch.  Since  there  is  no 
notch  there  is  practically  no  stress  augmentation  and  rela¬ 
tively  no  fatigue  damage.  As  such,  the  parameter  e  is  esti¬ 
mated  as 

max  (llp^-p'IIJ  (7) 

from  N  consecutive  observations  with  JV>  1 . 

The  algorithm  works  in  the  following  fashion:  the  refer¬ 
ence  point  p  is  initialized  to  the  starting  point  p°  and  anom¬ 
aly  measure  i j/°  is  set  to  0.  At  any  slow  time  epoch  tk  if  the 


state  probability  vector  moves  such  that  the  distance  trav¬ 
elled  in  any  particular  direction  (i.e.,  the  sup  norm  11*1100)  is 
greater  than  e  as  specified  in  step  (ii),  then  the  anomaly 
measure  is  incremented  by  Sil/k  =  d(pk .  p)  and  the  reference 
point  is  shifted  to  the  current  point  p/c.  The  procedure  is 
repeated  at  all  slow  time  epochs.  As  such,  the  total  path 
travelled  by  the  tip  of  probability  vector  represents  the 
deviation  from  the  nominal  condition  and  the  associated 
damage. 

3.4.  Real-time  implementation 

Fatigue  damage  monitoring  using  STSA  has  been  suc¬ 
cessfully  implemented  in  real  time.  The  nominal  condition 
is  chosen  after  the  start  of  the  experiment  at  time  epoch  t0, 
when  the  system  attains  a  steady  state  and  is  considered  to 
be  in  a  healthy  condition.  The  function  module  for  STSA  is 
triggered  at  this  point.  The  /)- Markov  machine  states  are 
fixed  in  advance  using  a  priori  determined  values  of 
the  parameters:  alphabet  size  |Xj  and  window  length  D. 
The  tasks  of  wavelet  space  partitioning  and  /i-Markov 
machine  construction  are  performed  based  on  the  time  series 
data  at  the  slow  time  epoch  t0  (nominal  condition).  The 
state  probability  vector  p°  at  time  epoch  t0  is  stored  for 
computation  of  anomaly  measures  at  subsequent  slow  time 
epochs,  1 1,  t2, . . .,  tk, . . .,  which  are  chosen  to  be  separated  by 
uniform  intervals  of  time  in  these  experiments.  The  ultra¬ 
sonic  data  files  at  time  epochs  .  .,tk, . . .  are  read  by 
the  STSA  function  module  that  calculates  the  anomaly 
measure  values  at  these  time  epochs.  The  algorithm  is  com¬ 
putationally  very  fast  (i.e.,  several  orders  of  magnitude  fas¬ 
ter  relative  to  slow  time  scale  damage  monitoring)  and  the 
evolution  of  anomaly  measure  is  exhibited  in  real  time.  The 
plot  is  updated  with  the  most  recent  value  of  anomaly  mea¬ 
sure  at  each  (slow  time)  epoch.  Thus,  the  STSA  algorithm 
allows  on-line  health  monitoring  and  is  capable  of  issuing 
warnings  of  incipient  failures  well  in  advance. 

3.5.  Advantages  of  STSA 

After  having  discussed  the  underlying  principles  and 
essential  features  of  STSA,  the  major  advantages  of  STSA 
for  anomaly  detection  are  listed  below: 

(a)  Robustness  to  measurement  noise  and  spurious  signals 
[31]  -  The  procedure  of  STSA  is  robust  to  measure¬ 
ment  noise  and  spurious  disturbances  and  it  filters 
out  the  noise  at  different  steps.  First  of  all,  coarse 
graining  of  the  continuous  data  (i.e.,  partitioning  into 
finite  blocks)  and  generation  of  a  symbol  sequence 
eliminate  small  measurement  noise  [28].  Secondly, 
the  wavelet  transform  also  contributes  in  signal-noise 
separation  of  the  raw  time  series  data  by  proper 
choice  of  scales  [31].  Finally,  the  state  probabilities 
are  generated  by  passing  a  long  symbol  sequence  over 
the  finite  state  machine,  which  further  eliminates 
small  (zero-mean)  measurement  noise. 
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(b)  Adaptability  to  low-resolution  sensing  due  to  coarse 
graining  in  space  partitions  [28], 

(c)  Capability  for  early  detection  of  anomalies  because  of 
sensitivity  to  signal  distortion  and  real-time  execution 
on  commercially  available  inexpensive  platforms 
[30,29]. 

(d)  Applicability  to  networked  communication  systems 
due  to  the  capability  of  data  compression  into  low¬ 
dimensional  pattern  vectors. 


3.6.  Summary  of  STS  A  based  anomaly  detection 

The  symbolic  time  series  analysis  (STSA)  procedure  of 

anomaly  detection  is  summarized  below. 

•  Acquisition  of  time  series  data  from  appropriate  sen- 
sor(s)  at  time  epoch  t0  of  the  nominal  condition,  when 
the  system  is  assumed  to  be  in  the  healthy  state  (i.e.,  zero 
anomaly  measure). 

•  Generation  of  the  wavelet  transform  coefficients, 
obtained  with  an  appropriate  choice  of  the  wavelet  basis 
and  scale  range. 

•  Maximum  entropy  partitioning  of  the  wavelet  scale  ser¬ 
ies  data  at  the  nominal  condition  (see  Section  3.1);  and 
generation  of  the  corresponding  symbol  sequence.  The 
partitioning  is  fixed  for  subsequent  time  epochs. 

•  Construction  of  the  Z>-Markov  machine  states  from  the 
symbol  alphabet  size  |T  and  the  window  length  I).  and 
generation  of  the  state  probability  vector  p"  at  time 
epoch  t0. 

•  Generation  of  time  series  data  sequences  at  subsequent 
slow  time  epochs,  t\,t2,-  ■  -Jk, ■  ■  ■,  and  their  conversion 
to  the  wavelet  domain  to  generate  respective  symbolic 
sequences  based  on  the  partitioning  at  time  epoch  t0. 

•  Generation  of  the  state  probability  vectors 

p',p2 _ ,pA,. . .  at  slow  time  epochs,  t\,t2,.  ■  ■, C, •  •  ■  from 

the  respective  symbolic  sequences  using  the  finite  state 
machine  constructed  at  time  epoch  t0. 

•  Computation  of  scalar  Anomaly  Measures 

. . .,  i jik,.  ■  ■  at  time  epochs,  t\,  t2,  ...,tk,... 


4.  Experimental  apparatus  and  sensors  for  damage  detection 

The  experimental  apparatus,  shown  in  Fig.  3,  is  a  spe¬ 
cial-purpose  uniaxial  fatigue  testing  machine,  which  is 
operated  under  load  control  or  strain  control  at  speeds 
up  to  12.5  Hz;  a  detailed  description  of  the  apparatus 
and  its  design  specifications  are  reported  in  [41].  The  test 
specimens  are  subjected  to  tensile-tensile  cyclic  loading 
by  a  hydraulic  cylinder  under  the  regulation  of  computer- 
controlled  electro-hydraulic  servo-valves.  The  damage 
estimation  and  life  prediction  subsystem  consists  of  data 
analysis  software  and  the  associated  computer  hardware. 
The  process  instrumentation  and  the  control  module  of 
the  fatigue  test  apparatus  are  briefly  described  below. 


Ultrasonic 

Transduca's 


Specimai 


Hydraulic 

Cylinda 


Microscope 


Fig.  3.  Computer-instrumented  apparatus  for  fatigue  testing. 


•  Closed  loop  servo-hydraulic  unit  and  controller.  The 
instrumentation  and  control  of  the  computer-controlled 
uniaxial  fatigue  test  apparatus  includes  a  load  cell,  an 
actuator,  the  hydraulic  system,  and  the  controller.  The 
servo-hydraulic  unit  can  provide  either  random  loads 
or  random  strains  to  a  specimen  for  both  low-cycle 
and  high-cycle  fatigue  tests  at  variable  amplitudes  and 
multiple  frequencies.  The  control  module  is  installed 
on  a  computer  which  is  dedicated  to  machine  operation. 
The  controller  runs  the  machine  according  to  a  schedule 
file  which  contains  the  loading  profile  and  the  number  of 
load  cycles.  The  real  time  data  from  the  extensometer 
and  load  cell  are  supplied  to  the  controller  for  operation 
under  specified  position  and  load  limits. 

•  Subsystem  for  data  acquisition,  signal  processing,  and 
engineering  analysis'.  In  addition  to  the  computer  for 
controlling  the  load  frame,  a  second  computer  is  used 
for  real-time  image  data  collection  from  the  microscope 
to  monitor  the  growth  of  surface  cracks.  This  computer 
controls  the  movement  of  the  microscope  to  focus  on 
the  region  of  the  crack  tip.  The  instrumentation  for 
ultrasonic  flaw  detection  scheme  is  connected  to  a  third 
computer.  The  ultrasonic  data  collected  on  this  com¬ 
puter  in  real-time  is  then  transferred  at  regular  intervals 
to  a  fourth  computer  on  which  the  STSA  software  is 
installed.  This  computer  performs  the  real-time  data 
analysis  task.  These  laboratory  computers  are  intercon¬ 
nected  by  a  local  dedicated  network  for  data  acquisition, 
data  communications,  and  control. 

Fig.  4  shows  a  typical  7075-T6  aluminum  specimen  used 
for  testing  in  the  fatigue  damage  test  apparatus.  The  spec¬ 
imens  used  are  3  mm  thick  and  50  mm  wide,  and  have  a 
slot  of  1.58  mm  x  4.5  mm  at  the  center.  The  central  notch 
is  made  to  increase  the  stress  concentration  factor  that 
ensures  crack  initiation  and  propagation  at  the  notch  ends. 
The  test  specimens  have  been  subjected  to  sinusoidal  load¬ 
ing  under  tension-tension  mode  (i.e.,  with  a  constant  posi- 
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Fig.  4.  Cracked  specimen  with  a  central  notch. 


tive  offset)  at  a  frequency  of  12.5  Hz.  The  DC  offset  was 
provided  in  the  load  cycling  to  ensure  that  the  specimen 
was  always  under  tension.  Since  inclusions  and  flaws  are 
randomly  distributed  across  the  material,  small  cracks 
appear  at  these  defects  and  propagate  and  join  at  the 
machined  surface  of  the  notch  even  before  microscopically 
visual  cracks  appear  on  the  surface. 

The  test  apparatus  is  equipped  with  two  types  of  sensors 
that  have  been  primarily  used  for  damage  detection: 

(1)  Travelling  optical  microscope'.  The  travelling  optical 
microscope,  shown  as  part  of  the  test  apparatus  in 
Fig.  3,  provides  direct  measurements  of  the  visible 
portion  of  a  crack.  The  resolution  of  the  optical 
microscope  is  about  2  pm  at  a  working  distance  of 
10-35  cm  and  the  images  are  taken  at  a  magnification 
of  75x.  The  growth  of  the  crack  is  monitored  contin¬ 
uously  by  the  microscope  which  takes  the  images  of 
the  surface  of  the  specimen  at  regular  intervals.  The 
microscope  shifts  from  left  to  right  side  of  the  central 
notch  and  vice  versa  after  every  200  cycles  to  track 
crack  growth  on  both  sides  of  the  notch.  In  order 
to  take  pictures  the  controller  slows  down  the 
machine  to  less  than  5  Hz  to  get  a  better  resolution 
of  the  images.  The  crack  length  can  be  calculated 
automatically  by  movement  of  the  microscope  from 
the  respective  notch  end  to  the  tip  of  the  crack.  The 
data  acquisition  software  also  allows  for  manual 
operation  and  image  capture  at  the  desired  moment. 
Formation  of  very  small  cracks  is  difficult  to  detect 
and  model  due  to  large  variability  of  material  irregu¬ 
larities.  This  paper  primarily  focuses  on  analyzing 
ultrasonic  data  for  more  accurate  characterization 
of  the  nature  of  small  defects. 

(2)  Ultrasonic  flaw  detector.  A  piezoelectric  transducer  is 
used  to  inject  ultrasonic  waves  in  the  specimen  and  an 
array  of  receiver  transducers  is  placed  on  the  other 
side  of  notch  to  measure  the  transmitted  signal.  In 
these  experiments,  an  array  of  2  receiver  transducers 
was  placed  below  the  notch  to  detect  faults  on  both 
left  and  right  side  of  the  notch.  The  ultrasonic  waves 
produced  were  5  MHz  sine  wave  signals  and  they 
were  emitted  during  a  very  short  portion  at  the  peak 
of  every  load  cycle.  Ultrasonic  measurements  were 
taken  at  stress  levels  that  exceeded  the  crack  opening 


Sender  Receiver 


stress  and  this  causes  maximum  attenuation  of  the 
ultrasonic  waves.  Note  that  if  crack  closure  occurs 
at  low-loads,  then  an  alternative  method  would  be 
needed  to  detect  anomalies.  The  sender  and  receiver 
ultrasonic  transducers  are  placed  on  two  positions, 
above  and  below  the  notch,  so  as  to  send  the  signal 
through  the  region  of  crack  propagation  and  receive 
it  on  the  other  side,  as  seen  in  Fig.  5. 

As  with  the  propagation  of  any  wave,  it  is  possible  that 
discontinuities  in  the  propagation  media  will  cause  additive 
and  destructive  interference.  Since  material  characteristics 
(e.g.,  voids,  dislocations  and  short  cracks)  influence  ultra¬ 
sonic  impedance,  a  small  fault  in  the  specimen  is  likely  to 
change  the  signature  of  the  signal  at  the  receiver  end.  The 
effect  of  these  discontinuities  in  the  material  is  to  distort 
the  transmitting  ultrasonic  waves.  Since  ultrasonic  waves 
have  a  very  small  wavelength,  very  small  faults  can  be 
detected.  Therefore,  the  received  signal  can  be  used  to  cap¬ 
ture  minute  details  and  small  changes  during  the  early 
stages  of  fatigue  damage,  which  are  not  possible  to  detect 
by  an  optical  microscope  [9].  Prior  to  the  appearance  of  a 
single  large  crack  on  the  surface  of  the  specimen  as  detected 
by  the  optical  microscope,  deformations  (e.g.,  dislocations 
and  short  cracks)  can  cause  detectable  attenuation  and/or 
distortion  of  the  ultrasonic  waves  [25],  Recent  literature 
has  also  shown  nonlinear  modelling  approaches  of  the 
ultrasonic  interference  with  the  material  micro-structures 
[42,43].  An  elaborate  description  of  the  properties  of  ultra¬ 
sonic  waves  in  solid  media  is  provided  by  Rose  [44], 

The  advantages  of  using  ultrasonic  flaw  detection  over  a 
microscope  are  the  ease  of  installation  at  the  desired  dam¬ 
age  site  and  detection  of  early  anomalies  before  the  onset  of 
widespread  fatigue  crack  propagation.  It  is  observed  that  a 
crack  always  starts  at  the  stress-concentrated  region  near 
the  notch  but  the  exact  site  of  crack  nucleation  can  be  trea¬ 
ted  as  a  random  event.  An  optical  microscope  is  only  capa¬ 
ble  of  detecting  cracks  when  they  appear  on  the  front 
surface  of  the  specimen.  Therefore,  the  study  in  this  paper 
is  based  on  analyzing  the  ultrasonic  data  for  identification 
of  fatigue  damage  in  the  small  crack  regime. 

5.  Experimentation  and  application  of  STSA  for  fatigue 
damage  monitoring 

The  fatigue  tests  were  conducted  on  center  notched 
specimens  with  the  objective  of  real-time  continuous  mon¬ 
itoring  of  fatigue  damage  growth  on  both  sides  of  the 
notch.  The  tests  were  performed  at  12.5  Hz  frequency 
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under  two  different  types  of  loading  conditions:  (a)  stress 
controlled  low-cycle  fatigue  and  (b)  stress  controlled  high- 
cycle  fatigue.  For  low-cycle  fatigue  loading  the  specimens 
were  subjected  to  a  sinusoidal  load  where  the  maximum 
and  minimum  loads  were  kept  constant  at  92.5  MPa  and 
4.85  MPa.  For  high-cycle  fatigue  loading  the  maximum 
and  minimum  amplitudes  were  kept  constant  at  71  MPa 
and  4.85  MPa. 

The  optical  images  were  collected  automatically  at  every 
200  cycles  until  a  crack  was  detected  on  the  specimen  sur¬ 
face  by  the  optical  microscope.  Subsequently,  the  images 
were  taken  at  user  command  and  the  microscope  was 
moved  such  that  it  always  focused  on  the  crack  tip.  A  sig¬ 
nificant  amount  of  micro-structural  damage  caused  by 
multiple  small  cracks,  dislocations  and  other  defects  occurs 
before  a  single  large  crack  appears  on  the  surface  of  the 
specimen  when  it  is  observed  by  the  optical  microscope 
[45].  This  phenomenon  causes  distortion  and  attenuation 
of  the  ultrasonic  signal  at  the  receiver  end.  The  crack  prop¬ 
agation  stage  starts  when  this  micro-structural  damage 
eventually  develops  into  a  single  large  crack.  Subsequently, 
the  crack  growth  rate  increases  rapidly  and  when  the  crack 
is  sufficiently  large,  complete  attenuation  of  the  transmitted 
ultrasonic  signal  occurs,  as  seen  at  the  receiver  end.  After 


the  crack  appears  on  the  surface,  fatigue  damage  growth 
can  be  easily  monitored  by  the  microscope  but  the  ultra¬ 
sonics  provide  early  warnings  even  during  the  crack  initia¬ 
tion  phase. 

Ultrasonic  waves  with  a  frequency  of  5  MHz  were  trig¬ 
gered  at  each  peak  of  the  sinusoidal  load  to  obtain  100  data 
points  in  each  cycle.  Since  the  ultrasonic  frequency  is  much 
higher  than  the  load  frequency,  data  acquisition  was  done 
for  a  very  short  interval  in  the  time  scale  of  load  cycling. 
Therefore,  it  can  be  implied  that  ultrasonic  data  were  col¬ 
lected  at  the  peak  of  each  sinusoidal  load  cycle,  where 
the  stress  is  maximum  and  the  crack  is  open  causing  max¬ 
imum  attenuation  of  the  ultrasonic  waves.  The  slow  time 
epochs  were  chosen  to  be  1000  load  cycles  (i.e.,  ~80  s) 
apart.  At  the  onset  of  each  slow  time  epoch,  the  ultrasonic 
data  points  were  collected  on  the  fast  time  scale  of  100 
cycles  (i.e.,  ~8  s),  which  produced  a  string  of  10,000  data 
points.  It  is  assumed  that  during  this  fast  time  scale  of 
100  cycles,  the  system  remained  in  a  stationary  condition 
and  no  major  changes  occurred  in  the  fatigue  damage 
behavior.  This  set  of  time  series  data  collected  in  the  man¬ 
ner  described  above  at  different  slow  time  epochs  was  ana¬ 
lyzed  using  the  STSA  method  to  calculate  the  anomaly 
measures  at  respective  slow  time  epochs. 


d)  Appearance  of  Surface  e)  Fully  Developed  f)  Broken  Specimen 

Crack  ~54  kilocycles  Crack  ~65  kilocycles  '-80  kilocycles 


Fig.  6.  Evolution  of  surface  crack  and  probability  distribution  on  left  side  of  the  notch  under  low  cycle  fatigue. 
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Following  the  STSA  procedure  for  anomaly  detection  as 
described  in  Section  3,  the  alphabet  size  and  depth  have 
been  chosen  to  be  |T|  =  8  and  Dm  1,  respectively;  conse¬ 
quently,  the  number  of  machine  states  is  2’|  D  =  8.  The 
wavelet  basis  has  been  chosen  to  be  ‘gaus2’  [46].  Absolute 
values  of  the  wavelet  scale  series  data  have  been  used  to 
generate  the  partition  because  of  the  symmetry  of  the  data 
sets  about  their  mean.  With  this  selection,  the  STSA  tool 
has  been  able  to  capture  the  anomalies  significantly  earlier 
than  the  optical  microscope.  Increasing  the  value  of  |T|  fur¬ 
ther  did  not  improve  the  results  and  increasing  the  value  of 
D  created  a  large  number  of  states  of  the  finite  state 
machine,  many  of  them  having  very  small  or  zero  probabil¬ 
ities,  and  required  a  larger  number  of  data  points  at  each 
time  epoch  to  stabilize  the  state  probability  vectors.  The 
advantage  of  having  a  small  number  of  states  is  fast  com¬ 
putation  on  inexpensive  processors  and  also  robustness 
to  noise.  The  wavelet  basis,  ‘gaus2’,  provides  better  results 
than  the  wavelet  bases  of  the  Daubechies  family  [38] 
because  the  ‘gaus2’  wavelet  base  closely  matches  the  shape 
of  the  sinusoidal  ultrasonic  signals. 

The  nominal  condition  at  the  slow  time  epoch  t0  was 
chosen  to  be  ~0.5  kilocycles  to  ensure  that  the  electro- 
hydraulic  system  of  the  test  apparatus  had  come  to  a  steady 


state  and  that  no  significant  damage  occurred  till  that 
point.  This  nominal  condition  was  chosen  as  a  benchmark 
where  the  specimen  was  assumed  to  be  in  a  healthy  state, 
and  thus  the  anomaly  measure  was  chosen  to  be  zero. 
The  anomalies  at  subsequent  slow  time  epochs, 
■  ■  ■,  tk,--  ■,  were  then  calculated  with  respect  to  the 
nominal  condition  at  t0.  It  is  emphasized  that  the  anomaly 
measure  is  relative  to  the  nominal  condition  which  is  fixed 
in  advance  and  should  not  be  confused  with  the  actual 
damage  at  an  absolute  level.  However,  inferring  fatigue 
damage  from  the  observed  anomaly  measure  is  an  inverse 
problem  that  is  a  topic  of  future  research. 

6.  Experimental  results  and  discussion 

This  section  presents  the  results  of  symbolic  time  series 
analysis  of  the  ultrasonic  data  generated  by  fatigue  tests 
described  in  Section  5.  In  each  of  the  six  plate  pairs  from 
(a)  to  (f)  in  Figs.  6-9,  the  top  plate  displays  the  surface 
image  of  the  test  specimen,  as  seen  by  the  optical  micro¬ 
scope;  and  histograms  of  the  state  probability  distribution 
in  the  bottom  plates  exhibit  evolution  of  fatigue  damage 
patterns  at  different  slow  time  epochs.  These  patterns 
gradually  change  from  uniform  distribution  (i.e.,  minimal 


a)  Nominal  Condition 
~0.5  kilocycles 


b)  Internally  Damaged 
Condition  —20  kilocycles 


c)  Internally  Damaged 
Condition  ~45  kilocycles 


d)  Appearance  of  Surface  e)  Fully  Developed 

Crack  —52  kilocycles  Crack  —65  kilocycles 


f)  Broken  Specimen 
-80  kilocycles 


Fig.  7.  Evolution  of  surface  crack  and  probability  distribution  on  right  side  of  the  notch  under  low  cycle  fatigue. 
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a)  Nominal  Condition 
~0.5  kilocycles 


b)  Internally  Damaged 
Condition  -50  kilocycles 


c)  Internally  Damaged 
Condition  —150  kilocycles 


d)  Appearance  of  Surface  e)  Fully  Developed  Crack 

Crack— 201  kilocycles  —220  kilocycles 


f)  Broken  Specimen 
~240  kilocycles 


Fig.  8.  Evolution  of  surface  crack  and  probability  distribution  on  left  side  of  the  notch  under  high-cycle  fatigue. 


information)  to  delta  distribution  (i.e.,  maximum  informa¬ 
tion)  in  each  figure. 

The  image  in  each  of  Figs.  6(a),  7(a),  8(a)  and  9(a)  shows 
the  nominal  condition  at  —0.5  kilocycles  when  the  anomaly 
measure  is  taken  to  be  zero.  This  is  considered  to  be  the  ref¬ 
erence  point  with  the  available  information  on  potential 
damage  being  minimal.  This  is  reflected  in  the  uniform  dis¬ 
tribution  (i.e.,  maximum  entropy  or  highest  uncertainty)  as 
seen  from  the  histograms  in  the  corresponding  bottom 
plates. 

The  images  in  plate  pairs  (b)  and  (c)  in  Figs.  6-9  show 
the  specimen  surface  before  crack  propagation.  These 
images  do  not  yet  have  any  indication  of  surface  crack 
although  the  corresponding  bottom  plates  do  exhibit  devi¬ 
ations  from  the  uniform  probability  distribution  (see  the 
bottom  plate  in  plate  pair  (a)  for  comparison).  This  is  an 
evidence  that  the  analytically  derived  results,  based  on 
ultrasonic  sensor  data,  produce  relevant  damage  informa¬ 
tion  during  crack  initiation;  this  information  is  not  avail¬ 
able  from  the  corresponding  optical  images.  For  low- 
cycle  fatigue  in  Figs.  6  and  7,  the  plate  pairs  (b)  and  (c) 
represent  damage  status  at  —20  and  —45  kilocycles,  respec¬ 
tively;  and  for  high-cycle  fatigue  in  Figs.  8  and  9,  they  rep¬ 
resent  damage  status  at  —50  and  —  150  kilocycles, 


respectively.  The  deviations  of  these  histograms  from  those 
at  the  nominal  condition  in  plates  (a)  indicate  that  the 
probability  vector  has  moved  to  another  location  on  the 
surface  of  unity-radius  hypersphere.  The  path  depends  on 
the  interactions  of  the  ultrasonic  signals  with  the  subsur¬ 
face  deformities. 

The  image  in  each  of  Figs.  6(d),  7(d),  8(d)  and  9(d) 
exhibits  the  first  noticeable  appearance  of  a  crack  on  the 
specimen  surface,  which  is  often  deemed  as  the  boundary 
of  crack  initiation  and  crack  propagation  phases;  however, 
crack  propagation  might  have  started  before  the  appear¬ 
ance  of  a  surface  crack.  For  low-cycle  fatigue  in  Figs.  6 
and  7,  surface  cracks  have  been  detected  by  the  microscope 
at  —54  and  —52  kilocycles  on  the  left  side  and  right  side  of 
the  notch,  respectively.  Similarly,  for  high-cycle  fatigue  in 
Figs.  8  and  9,  surface  cracks  have  been  detected  at  —201 
and  —215  kilocycles  on  left  side  and  right  side  of  the  notch, 
respectively.  The  appearance  of  a  large  surface  crack  indi¬ 
cates  that  a  significant  portion  of  the  crack  or  multiple 
small  cracks  might  have  already  developed  underneath 
the  surface  before  they  started  spreading  on  the  surface. 
However,  further  micro-structural  analysis  is  need  to  con¬ 
firm  these  findings.  The  histogram  of  probability  distribu¬ 
tion  in  the  corresponding  bottom  plates  show  further 
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a)  Nominal  Condition 
-0.5  kilocycles 


b)  Internally  Damaged 
Condition  ~50  kilocycles 


c)  Internally  Damaged 
Condition  — 150  kilocycles 


d)  Appearance  of  Surface  e)  Fully  Developed  f)  Broken  Specimen 

Crack  ~215  kilocycles  Crack  -230  kilocycles  ~240  kilocycles 


Fig.  9.  Evolution  of  surface  crack  and  probability  distribution  on  right  side  of  the  notch  under  high-cycle  fatigue. 


evolution  of  the  probability  vector  with  the  fatigue  dam¬ 
age.  At  this  stage,  the  information  on  damage  is  enhanced 
compared  to  what  was  available  in  the  earlier  cycles.  At  the 
onset  of  the  crack  propagation  phase,  the  histograms  of 
probability  distribution  show  drastic  changes  in  patterns, 
which  indicate  rapid  development  of  large  cracks. 

The  image  in  each  of  Figs.  6(e),  7(e),  8(e)  and  9(e)  exhib¬ 
its  a  fully  developed  crack  in  its  propagation  phase.  The 
corresponding  bottom  plate  shows  the  histogram  of  the 
probability  distribution  that  is  significantly  different  from 
those  in  earlier  cycles  in  plate  pairs  (a)  (d),  which  shows 
further  gain  in  the  information  on  crack  damage.  The 
image  in  each  of  Figs.  6(f)  and  7(f)  exhibits  a  completely 
broken  specimen  at  ~80  kilocycles  for  low-cycle  fatigue. 
Similarly,  the  image  in  each  of  Figs.  8(f)  and  9(f)  exhibits 
a  completely  broken  specimen  at  ~240  kilocycles  for 
high-cycle  fatigue.  The  corresponding  bottom  plates  show 
the  delta  distribution  indicating  complete  information  on 
crack  damage. 

The  observation  in  Figs.  6-9  is  further  clarified  by  using 
the  notion  of  entropy  (see  Eq.  (2)).  The  data  at  the  nominal 
condition  have  been  partitioned  using  the  maximum 
entropy  principle,  which  leads  to  uniform  probability  dis¬ 
tribution  as  seen  in  the  bottom  plate  of  plate  pair  (a)  in 


Figs.  6-9.  In  contrast,  for  the  completely  broken  stage  of 
the  specimen,  the  entire  probability  distribution  is  concen¬ 
trated  in  only  one  state  as  seen  in  the  bottom  plate  of  plate 
pair  (f)  in  Figs  6-9,  due  to  very  large  attenuation  of  the 
ultrasonic  signal.  This  phenomenon  of  the  test  specimen 
being  completely  broken  signifies  certainty  of  information 
and  hence  zero  entropy.  Therefore,  as  the  fatigue  crack 
damage  evolves,  the  uniform  distribution  (i.e.,  maximum 
entropy)  under  nominal  condition  degenerates  toward  the 
delta  distribution  (i.e.,  zero  entropy)  for  the  broken  speci¬ 
men.  In  the  intermediate  stages,  gradual  degradation  can 
be  quantitatively  evaluated  using  this  information. 

Fig.  10  shows  the  evolution  of  anomaly  measure  profiles 
obtained  by  STSA  of  ultrasonic  data  for  low-cycle  and 
high-cycle  fatigue  and  the  corresponding  crack  growth 
plots.  The  top  plate  on  the  left  column  in  Fig.  10  shows 
profiles  of  anomaly  measure  for  both  left  and  right  sides 
of  the  notch  under  low-cycle  fatigue.  Similarly,  the  top 
plate  on  the  right  column  shows  profiles  of  anomaly  mea¬ 
sure  for  both  left  and  right  sides  of  the  notch  under  high- 
cycle  fatigue.  The  bottom  plates  show  the  corresponding 
crack  growth  profiles.  In  each  curve  in  the  top  plates  of 
Fig.  10,  the  sharp  change  in  the  slope  of  the  anomaly  mea¬ 
sure  indicates  the  onset  of  crack  propagation  phase.  This 
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Kilocycles  Kilocycles 

Fig.  10.  Profiles  of  anomaly  measure  and  crack  growth  for  center-notched  specimens  under  low  cycle  and  high-cycle  fatigue. 


occurs  approximately  upon  appearance  of  a  surface  crack. 
The  vertical  dashed  lines  in  the  top  two  plates  of  Fig.  10 
indicate  the  first  detection  of  a  surface  crack  by  the  optical 
microscope. 

It  is  observed  that  small  changes  can  be  detected  by 
STSA  significantly  before  the  microscope  can  capture  a 
surface  crack.  The  slope  of  the  anomaly  measure  represents 
the  fatigue  damage  growth  rate  while  the  magnitude  indi¬ 
cates  the  accumulated  fatigue  damage  starting  from  the 
nominal  condition.  An  abrupt  increase  in  the  slope  (i.e., 
a  sharp  rise  in  the  curvature)  of  anomaly  measure  profile 
provides  a  clear  insight  into  a  forthcoming  failure.  The 
region  to  the  right  of  the  vertical  dashed  lines  for  left  side 
and  right  side  anomaly  measure  curves  in  the  top  plate 
on  left  column  of  Fig.  10  can  be  considered  to  be  the 
boundary  of  the  crack  initiation  and  crack  propagation 
phases,  where  the  growth  of  anomaly  measure  is  signifi¬ 
cantly  faster  than  that  in  the  crack  initiation  phase.  How¬ 
ever,  the  critical  information  lies  in  the  region  towards 
the  left  of  the  vertical  lines  which  is  broadly  identified  as 
crack  initiation  phase  where  no  crack  was  visible  on  the 
surface.  This  is  the  region  where  micro-structural  damage 
such  as  multiple  small  cracks  possibly  caused  small  changes 


in  the  ultrasonic  signal  profile.  After  the  multiple  small 
cracks  coalesce  together  to  form  a  single  large  crack,  the 
crack  propagation  phase  starts.  Damage  evolution  under¬ 
goes  elastoplastic  deformation  under  low-cycle  fatigue 
due  to  a  large  number  of  dislocation  movements  and  multi¬ 
ple  small  cracks  develop  from  a  very  early  stage.  Similar 
effects  are  observed  for  high-cycle  fatigue  as  seen  in  the 
top  plate  on  right  column  of  Fig.  10.  In  the  region  towards 
the  right  of  the  vertical  dashed  lines,  anomaly  measure  pro¬ 
files  show  significantly  faster  growth  as  compared  to  the 
slow  growth  during  crack  initiation  towards  the  left  of 
the  vertical  lines.  For  low-cycle  fatigue,  a  relatively  large 
slope  in  the  anomaly  measure  is  observed  even  during 
crack  initiation  region  indicating  occurrence  of  significant 
damage  during  this  phase.  Crack  initiation  prevailed  for 
~52-55  kilocycles  for  low-cycle  fatigue,  which  is  ~68%  of 
the  total  life.  For  high-cycle  fatigue,  crack  initiation  pre¬ 
vailed  for  a  relatively  longer  period  of  ~205-215  kilocycles 
for  development  of  a  single  large  crack,  which  amounts  to 
~87%  of  the  total  life. 

The  top  two  plates  in  Fig.  10  show  a  relatively  large 
slope  of  anomaly  measure  from  the  start  of  cyclic  loading 
to  ~5  kilocycles  for  both  low-cycle  and  high-cycle  fatigue. 
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This  is  the  stage  where  micro-structural  damage  (e.g.,  due 
to  dislocation  movements  and  accumulation,  and  persistent 
slip  band  formation)  induces  hardening  of  the  strained 
components  [45]  and  changes  the  ultrasonic  impedance 
leading  to  a  sharp  rise  in  the  local  slope  of  the  anomaly 
measure  profile.  After  these  initial  effects  subside,  a  modest 
reduction  of  the  slope  takes  place  for  the  remaining  part  of 
the  crack  initiation  phase.  A  sharp  rise  of  slope  is  again 
observed  at  the  onset  of  the  crack  propagation  phase.  Sim¬ 
ilar  phenomena  were  observed  by  Berkovits  and  Fang  [47] 
in  acoustic  emission  experiments  on  smooth  specimens  of 
Incoloy  901  at  room  temperature.  Further  experiments 
and  micro-structural  analysis  are  necessary  to  confirm 
these  findings. 

7.  Summary,  conclusions,  and  future  work 

This  paper  presents  the  concept,  theory,  and  experimen¬ 
tal  validation  of  a  statistical  pattern  identification  tool  for 
early  detection  and  online  monitoring  of  fatigue  damage  in 
polycrystalline  alloys.  The  underlying  principle  of  fatigue 
damage  detection  is  built  upon  symbolic  time  series  analy¬ 
sis  of  ultrasonic  sensor  signals.  A  combination  of  maxi- 
mum-entropy  partitioning  in  the  wavelet  domain  and 
symbolic  dynamics  enables  fatigue  damage  detection  signif¬ 
icantly  before  the  onset  of  crack  propagation.  The  codes  of 
damage  analysis  are  executable  in  real  time  and  have  been 
demonstrated  on  a  special-purpose  fatigue  testing  appara¬ 
tus  on  7075-T6  aluminum  alloy  specimens  under:  stress 
controlled  low-cycle  fatigue  and  stress  controlled  high- 
cycle  fatigue.  The  results  consistently  indicate  that  the  code 
is  capable  of  detecting  damage  before  any  surface  cracks 
are  captured  by  the  optical  microscope. 

The  reported  work  is  a  step  toward  building  a  reliable 
instrumentation  system  for  early  detection  of  fatigue  dam¬ 
age  in  polycrystalline  alloys.  Further  experimental,  analyt¬ 
ical  and  micro-structural  research  is  necessary  before  its 
usage  in  industry.  While  there  are  many  technical  issues 
that  need  to  be  addressed,  the  following  research  topics 
are  being  currently  pursued. 

•  Statistical  analysis  of  an  ensemble  of  ultrasonic  time  ser¬ 
ies  data  sets,  collected  under  identical  loading  and  envi¬ 
ronmental  conditions,  to  account  for  manufacturing  and 
material  uncertainties. 

•  Microstructural  analysis  to  investigate  the  early  stages 
of  fatigue  damage. 

•  Validation  of  the  STSA  technique  for  fatigue  damage 
monitoring  under  different  conditions,  such  as  vari¬ 
able-amplitude  block  loading  and  spectral  loading. 
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Abstract 

This  paper  investigates  correlation  properties  of  fluctuations  in  fatigue  crack  growth  of 
polycrystalline  materials,  such  as  ductile  alloys,  that  are  commonly  encountered  in  structures 
and  machinery  components  of  complex  electromechanical  systems.  The  model  of  crack  damage 
measure  indicates  that  the  fluctuations  of  fatigue  crack  growth  are  characterized  by  strong 
correlation  patterns  within  short-time  scales  and  are  uncorrelated  for  larger  time  scales.  The 
two  correlation  regimes  suggest  that  the  7075-T6  aluminum  alloy,  analyzed  in  this  paper,  is 
characterized  by  a  micro-structure  which  is  responsible  for  an  intermittent  correlated  dynamics 
of  fatigue  crack  growth  within  a  certain  scale.  The  constitutive  equations  of  the  damage 
measure  are  built  upon  the  physics  of  fracture  mechanics  and  are  substantiated  by 
Karhunen-Loeve  decomposition  of  fatigue  test  data.  Statistical  orthogonality  of  the  estimated 
damage  measure  and  the  resulting  estimation  error  are  demonstrated  in  a  Hilbert  space  setting. 
©  2005  Elsevier  B.V.  All  rights  reserved. 


1.  Introduction 

The  fracture  of  solids  and  the  growth  of  cracks  is  a  typical  instability  phenomena 
which  are  known  to  be  strongly  nonlinear.  Herein  we  apply  to  fracture  mechanics 
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some  of  the  recent  methods  developed  in  statistical  physics.  In  particular,  we  use  the 
notion  of  fractal  statistics  to  describe  the  correlation  of  the  fluctuations  around 
fatigue  crack  growth  in  polycrystalline  materials,  such  as  ductile  alloys.  In  this  paper, 
we  have  investigated  the  fatigue  fracture  properties  of  7075-T6  aluminum  alloy. 

The  importance  of  this  investigation  is  that,  in  both  physics  and  engineering 
literature,  the  fluctuations  around  fatigue  crack  growth  in  a  typical  material  have 
always  been  assumed  to  be  random  or  uncorrelated  noise.  Consequently,  the 
associated  models  include  uncorrelated  random  processes.  For  example,  in 
agreement  with  the  existent  theory  of  micro-level  fatigue  cracking,  Bogdanoff  and 
Kozin  [1]  proposed  a  Poisson-like  uncorrelated-increment  jump  model  of  fatigue 
crack  phenomena.  An  alternative  approach  to  stochastic  modeling  of  fatigue  crack 
damage  is  to  randomize  the  coefficients  of  an  existing  deterministic  model  to 
represent  material  inhomogeneity  [2],  A  third  approach  has  been  to  adopt  a 
deterministic  model  of  fatigue  crack  growth  in  addition  to  a  random  process,  see  for 
example  Refs.  [3-5]. 

The  fatigue  crack  growth  process  can  also  be  modeled  by  nonlinear  stochastic 
differential  equations  using  Ito  statistics  [6]  that  again  presuppose  randomness  of  the 
fluctuations.  Specifically,  the  Kolmogorov  forward  and  backward  diffusion 
equations,  which  require  solutions  of  nonlinear  partial  differential  equations,  have 
been  proposed  to  generate  the  statistical  information  required  for  risk  analysis  of 
mechanical  structures  [7,8].  These  nonlinear  partial  differential  equations  have  only 
been  solved  numerically  and  the  numerical  procedures  are  computationally  intensive 
as  they  rely  on  fine-mesh  models  using  finite-element  or  combined  finite-difference 
and  finite-element  methods  [9],  Casciati  et  al.  [10]  have  analytically  approximated  the 
solution  of  the  Ito  equations  by  Hermite  moments  to  generate  a  probability 
distribution  function  of  the  crack  length. 

Several  studies  have  determined  that  the  stochastic  fluctuations  observed  in 
innumerable  natural  phenomena  are  not  simply  random,  i.e.,  uncorrelated  noise,  but 
present  correlation  patterns  that  reveal  complex  and  alternative  dynamics  and/or 
material  microstructures.  Thus,  the  purpose  of  the  present  research  is  to  determine 
whether  uncorrelated  stochastic  models  such  as  those  previously  discussed  in  the 
literature  are  realistic  in  describing  the  fluctuations  around  fatigue  crack  growth  in 
polycrystalline  materials,  or  whether  such  fluctuations  present  patterns  that  would 
reveal  complex  material  micro-structure  requiring  alternative  correlated  stochastic 
models.  Two  main  classes  of  correlation  patterns  are  commonly  observed  in  natural 
time  series  and  these  are  denoted  as  short-  and  long-time  correlations.  Short-time 
correlations  are  characterized  by  phenomena  that  rapidly  lose  memory  of  past  or 
distant  events.  This  happens,  for  example,  when  the  autocorrelation  function  of  the 
time  series  decays  exponentially  in  the  time  separation  between  two  elements.  By 
contrast,  long-time  correlations  are  characterized  by  autocorrelation  functions  that 
decay  more  slowly  than  (negative)  exponentials;  one  example  is  the  inverse  power- 
law  decay. 

A  simple  model,  which  has  been  extensively  used  in  the  interpretation  of  stochastic 
fluctuations  in  a  time  series  {£,•}  with  i  =  1, 2, . . . ,  N,  is  based  on  the  evaluation  of  the 
mean-square  displacement  of  the  diffusion-like  processes  generated  by  trajectories 
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X  n(t)  defined  as 


xn(t)  =  Y  £n+j  ■ 


(1) 


If  {£,}  is  a  white  random  sequence,  the  diffusion  process  is  a  well-known  Brownian 
motion.  The  central  limit  theorem  applied  to  the  diffusion  distribution  generated  by 
trajectories  X  n(t)  yields  a  probability  density  that  converges  to  a  Gaussian  function 
whose  mean-square  displacement  converges  asymptotically  to 


(X(t)2)  oc  f  , 


(2) 


with  a  =  1.  In  general,  it  is  possible  to  have  anomalous  behavior  yielding  enhanced 
diffusion  (a  >  1 )  that  has  been  known  for  20  years  to  arise  in  dynamically  chaotic 
systems  [11],  or  sublinear  diffusive  growth  (oc  <  1)  that  is  familiar  from  disordered 
fractal  materials  [12]. 

Anomalous  diffusion  reveals  persistent  (for  an  enhanced  diffusive  growth)  or 
antipersistent  (for  a  sublinear  diffusive  growth)  correlation  patterns  in  the  dynamics 
of  a  random  walk.  A  persistent  random  walk  is  characterized  by  a  probability  of 
stepping  in  the  direction  of  the  previous  step  that  is  greater  than  that  of  reversing 
directions.  An  antipersistent  random  walk  is  characterized  by  a  probability  of 
stepping  in  the  direction  of  the  previous  step  that  is  less  than  that  of  reversing 
directions.  Sometimes  a  momentarily  initial  enhanced  or  sublinear  diffusive  growth, 
lasting  up  to  a  certain  time-scale,  is  generated  by  the  statistical  transition  to  the 
asymptotic  regime  of  the  diffusion  process.  For  example,  a  simple  discrete  random 
walk  is  described  by  a  binomial  distribution  that  only  asymptotically  converges  to  a 
Gaussian  while  initially  presenting  an  enhanced  diffusive  growth  [13].  Thus,  a  real 
autocorrelated  time  series  will  lose  its  correlation  patterns  if  the  temporal  order  of 
the  sequence  is  randomized. 

There  are  a  number  of  different  theoretical  approaches  that  explain  the  anomalous 
diffusion  depicted  in  (2).  One  such  explanatory  model  is  that  of  an  infinitely  long 
correlated  random  walk  in  which  a  =  2 H,  where  H  is  the  Hurst  exponent  in  the 
interval  0^/Z^l  with  the  case  H  —  0.5  corresponding  to  a  simple  random  walk. 
This  model  has  been  used  extensively  in  the  interpretation  of  fluctuations  in  time 
series  in  the  physical  and  life  sciences  [14]  and  is  called  fractional  Gaussian  noise  [15]. 
Another  kind  of  anomalous  diffusion  has  to  do  with  taking  steps  that  are 
uncorrelated  in  time,  but  on  a  random  or  fractal,  not  a  regular  lattice.  In  the  second 
model,  an  anomalous  diffusion  occurs  because  geometrical  obstacles  exist  on  all 
length  scales  and  such  obstacles  inhibit  transport.  Havlin  and  Ben-Avraham  [12] 
point  out  that  the  anomalous  exponent  a  is  related  to  the  fractal  dimension  of  the 
random  walk  path  on  the  lattice.  There  is  a  third  possible  explanation  of  the 
anomaly  in  (2)  called  a  Levy  walk  [16]  that  was  first  used  to  understand  turbulent 
diffusion  [16]  and  yields  a  3,  which  is  consistent  with  Richardson’s  law  of 
enhanced  diffusion  [17]. 

Physical  examples  of  anomalous  diffusion  processes  are  earthquakes  [18],  rainfall 
[16,19],  turbulent  fluid  flow  [20],  relaxation  of  stress  in  viscoelastic  materials  [14,21], 
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solar  flares  [22-24],  and  other  processes  with  slip-stick  dynamics.  Recently,  a  multi¬ 
scaling  comparative  analysis  to  distinguish  Levy  walk  intermittent  noise  from  fractal 
Gaussian  intermittent  noise  was  suggested  by  Scafetta  and  West  [25]. 

Finally,  a  physical  system  might  be  characterized  by  different  values  of  the  scaling 
exponent  a  at  different  scales  [26].  Usually,  this  means  that  one  system  is 
characterized  by  a  non-self-affine  structure.  The  scale  at  which  the  transition  from 
one  scaling  regime  to  another  occurs  indicates  the  scale  at  which  the  structure 
changes.  In  this  work  we  determine  that  the  fluctuations  around  the  ballistic  growth 
of  fatigue  cracks  in  ductile  alloys  present  such  a  scale  transition  from  a  strongly 
correlated  regime  at  short-time  scales  to  a  random  regime  at  longer  time-scales. 
Properties,  such  as  grain  size  distribution,  degree  of  heterogeneity,  the  existence  of 
microscopic  defects,  inclusions,  twin  boundaries  and  dislocations,  of  polycrystalline 
materials  may  contribute  to  the  micro-mechanisms  of  fatigue  fracture  revealed  by 
the  present  analysis. 

This  paper  is  organized  into  six  sections,  including  the  present  one,  and  an 
appendix.  Section  2  provides  the  underlying  phenomenology  of  the  stochastic 
damage  measure.  Section  3  presents  Karhunen-Loeve  (KL)  decomposition  of 
fatigue  test  data  to  formulate  an  estimate  of  the  stochastic  measure,  which  is 
statistically  orthogonal  to  the  estimation  error.  Section  4  focuses  on  identification  of 
the  model  parameters  and  their  probability  distributions.  Section  5  presents  the 
results  of  model  prediction  by  Monte  Carlo  simulation.  The  paper  is  summarized 
and  concluded  in  Section  6  with  recommendations  for  future  research. 


2.  Measure  of  fatigue  crack  damage 

Traditionally  fatigue  crack  growth  models  have  been  formulated  by  fitting 
estimated  mean  values  of  fatigue  crack  length  atl  generated  from  ensemble  averages 
of  experimental  data,  as  functions  of  time  in  units  of  cycles  [27,28].  Ray  and 
Patankar  [29]  have  formulated  the  state-space  modeling  concept  of  crack  growth 
based  on  fracture-mechanistic  principles  of  the  crack-closure  concept  [30].  The  state- 
space  model  has  been  validated  by  fatigue  test  data  for  variable-amplitude  cyclic 
loading,  see  for  example  Refs.  [28,31,32], 

The  three  panels  in  Fig.  1  show  test  data  of  cumulative  fatigue  crack  growth  in  the 
7075-T6  aluminum  alloy  under  different  cyclic  loading  [33].  It  is  important  to  note 
that  the  crack  growth  curves  do  not  increase  smoothly,  but  they  exhibit  fluctuations 
around  an  ideal  smooth  curve  of  crack  growth  representing  ballistic  growth.  In  this 
context,  a  major  objective  of  the  paper  is  to  investigate  the  autocorrelation 
properties  of  these  fluctuations  with  the  smooth  curve  removed.  In  the  following  we 
briefly  review  the  theory  and  the  standard  phenomenological  equations  that  describe 
the  fatigue  crack  growth. 

In  linear  fracture  mechanics,  it  is  assumed  that  the  stressed  material  remains  elastic 
and  undamaged  everywhere,  except  in  a  small  domain  in  the  vicinity  of  the  crack  tip. 
However,  this  view  is  not  confirmed  by  experimental  evidence  and  the  process  of  fatigue 
damage  accumulation  could  occur  throughout  the  stressed  volume.  Paris  and  Erdogan 
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Fig.  1.  Experimental  data  of  7075-T6  aluminum  alloy,  (a)  R  =  0.6  and  Max  stress  =  70.65  MPa;  (b) 
R  =  0.6  and  Max  stress  =  69.00  MPa;  (c)  R  =  0.6  and  Max  stress  =  47.09  MPa. 
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[27]  originally  developed  a  phenomenological  model  of  crack  growth  rate,  which 
depends  on  the  stress  history  and  is  thus  represented  by  a  continuum  rate  equation 
having  the  hereditary  structure.  This  model  has  been  subsequently  modified  by  many 
researchers  (see,  for  example,  citations  in  Refs.  [34—36])  in  the  following  form: 

da,  =  a,  —  atst  =  h{AKef  )5t ,  (3) 

with  h(  0)  =  0  and  a,0  >  0  for  to,  where  a,  is  the  estimated  mean  of  the  crack  length  at 
time  t  during  a  stress  cycle  and  8t  is  the  time  duration  of  the  stress  cycle;  and  A Kef  is  the 
stress  intensity  factor  range  at  time  t,  which  is  given  by  the  experimentally  validated 
empirical  model 

A Kf  =  A Sty/nat_stF(at-St) ,  (4) 

where  AS,  is  the  range  (i.e.,  the  difference  between  maximum  and  minimum  values)  of 
the  stress  cycle  at  time  t,  which  is  directly  related  to  the  applied  load.  Experimental 
observations  suggest  that  both  duration  and  shape  of  a  stress  cycle  are  not  relevant  for 
crack  growth  in  ductile  alloys  at  room  temperature.  A  stress  cycle  is  only  characterized 
by  the  minimum  stress  S™”  and  the  maximum  stress  Smax,  respectively,  and  is  denoted 
as  the  ordered  pair  (S  ,  Amax).  The  empirical  relation  F(»)  in  Eq.  (4)  represents  the 
geometry  of  the  crack  tip;  for  center-cracked  specimens  of  half-width  w  with  0  <  a,  <  w 
at  all  t^to,  the  structure  of  F(»)  has  been  experimentally  determined  as  [35] 


F(at-st) 


(5) 


The  function  /;(•)  in  Eq.  (3)  is  a  non-negative  Lebesgue-measurable  function  that  is 
dependent  on  the  material  and  geometry  of  the  stressed  component.  It  has  been  shown 
in  the  fracture  mechanics  literature  [35,36]  that,  for  center-cracked  specimens  of  ductile 
alloys,  the  function  h(»)  obeys  the  power  law: 


h(AKf)  =  (A Kf)m  , 


(6) 


where  the  exponent  parameter  m  is  dependent  on  the  material  of  the  stressed 
component;  for  ductile  alloys,  m  is  in  the  range  of  2. 5-5.0  [35], 

Eqs.  (3)-(6)  are  now  combined  to  formulate  a  mean-value  model  of  fatigue  crack 
growth  for  center-cracked  specimens  of  ductile  alloy  materials: 


5a,  oc 


5t , 


(7) 


with  d,o>0  and  f5=?o- 

Following  Sobczyk  and  Spencer  [9]  and  the  pertinent  references  cited  therein,  we 
randomize  the  deterministic  mean- value  model,  Eq.  (7),  to  obtain  a  stochastic  model 
for  the  rate  of  crack  growth.  The  stochastic  model  of  continuous  crack  length  is  built 
upon  the  model  structure  proposed  by  Ray  [29,37],  and  is  given  by 


dc,(0  =  Q(C,t) 


A  5m 


ct(  0 


cos((7i/2)ct(Q) 


d  t  =  Q{C,t) 


(AS,y^(0r 

1  -  m((n/4)ct(0y 


dr,  (8) 
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where  the  random  sample  £  signifies  a  specimen  or  a  machine  component  on 
which  a  fatigue  test  is  conducted;  the  dimensionless  stochastic  crack  length 
c,(£)  is  normalized  with  respect  to  the  half  width  w,  i.e.,  the  mean  value  c,  =  at/xv. 
Eq.  (8)  is  a  continuous  stochastic  version  of  Eq.  (7),  where  the  differential 
of  the  stochastic  crack  length  dc,(C)  is  a  function  of  the  crack  length  c,(C) 
at  time  t  and  the  normalized  stress  AS,  =  A Set  /Sy,  where  Sy  is  the  yield  stress 
of  the  material.  The  condition  0<c,a^c,<A/ns/rn  is  imposed  to  ensure 
non-negativity  of  the  crack  length  increment  almost  surely,  i.e.,  dc,(£)>0  for 
almost  all  samples  £.  The  stochastic  process  of  crack  growth  is  largely  dependent 
on  the  second-order  random  process  f2(£,  t)  and  the  exponent  parameter  m  in 
Eq.  (8). 

To  investigate  the  stochastic  properties  of  the  fatigue  crack  growth  process,  we 
separate  Q(C,  t)  into  two  parts  as 

fi«,o  =  oo(0[i  +  fii(e,o]<  (9) 


where  the  time-independent  component  f20(£)  represents  uncertainties  in  manufac¬ 
turing,  for  example  in  machining,  and  makes  a  major  contribution  to  the  ballistic 
component  of  the  crack  growth;  the  time-dependent  component  f2i(£,  t)  represents 
uncertainties  in  the  material  micro-structure  and  crack  length  measurements  that 
may  vary  with  crack  propagation  in  a  sample  £.  This  latter  component  is  primarily 
responsible  for  the  small  fluctuations  around  the  ballistic  component  of  crack 
growth  whose  autocorrelation  properties  we  study. 

We  postulate  that  Qo  and  in  Eq.  (9)  are  statistically  independent  of  one  another 
for  all  t^to,  where  to  is  the  initial  time.  The  rationale  for  this  independence 
assumption  is  that  inhomogeneity  of  the  material  micro-structure  and  measurement 
noise,  associated  with  each  test  specimen  and  represented  by  Q\(£,  t),  are  unaffected 
by  the  uncertainty  Qq(0  due,  for  example,  to  machining  operations.  Without  loss  of 
generality,  we  assume  that  the  fluctuations  in  time  have  a  zero  mean  value,  i.e., 
(Qi(C,t))  —  0  for  all  t ^ to .  Furthermore,  non-negativity  of  the  crack  growth  rate 
dc,(C)  in  Eq.  (8)  is  assured  in  the  almost  sure  (a.s.)  sense  by  imposing  the  constraint 
C2o(0^0  with  probability  1  (w.p.  1). 

For  notational  brevity,  let  us  suppress  the  term  £  in  random  processes  like  c,(() 
and  £2(£,  t).  A  combination  of  Eqs.  (8)  and  (9)  and  few  simple  algebraic  steps  yield 
the  following  equation  for  each  sample  point  £: 


2— m/2 

C, 


dcf  =  (AS,)mG0[l  +fii(0]df 


w.p.  1  . 


(10) 


Pointwise  integration  of  Eq.  (10)  yields  the  solution  of  fatigue  damage  increment 
from  the  initial  time  f0  to  the  current  time  t  as 


(ASV)mi20[l  +  £21(0]df/ 


w.p.  1  . 


> Kt,  to)  = 


(ID 
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An  explicit  expression  of  the  stochastic  diffusion  process  ij/(t,  to)  is  obtained  by 
integrating  the  left-hand  side  of  Eq.  (10)  and  is  given  by 
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(12) 


where  i p(t,  to)  represents  a  dimensionless  non-negative  measure  of  fatigue  crack 
damage  increment  from  the  initial  instant  to  to  the  current  instant  t  as  a  function  of 
the  normalized  crack  length.  The  constant  parameter  m  in  (12)  is  in  the  range  of 
2.5-5  for  ductile  alloys  and  metallic  materials  ensuring  that  (1  —  m/2)<0  and 
(3  —  m/2)>0.  The  diffusion  process  \j/(t,  to)  is  almost  surely  continuous  because  it  is 
a  continuous  function  of  the  crack  length  process  c,  w.p.  1.  Both  c,  and  >l/(t,  t0)  are 
measurable  functions  although  their  (probability)  measure  spaces  are  different.  In 
essence,  the  probability  of  t jj(t,  to),  conditioned  on  the  initial  crack  length  ct(t,  leads  to 
a  stochastic  measure  of  fatigue  crack  damage  increment  at  the  instant  t  starting  from 
the  initial  instant  to. 

For  a  constant  stress  range  AS,  we  carry  out  the  time  integration  in  Eq.  (11)  to 
obtain 


>Ht,  to)  =  (AS)m[Q0(t  -  t0)  +  0(t,  ?0)] , 


(13) 


where  the  second  term  on  the  right-hand  side  is  the  time  integral 

0(t,  to)  =  Qo  f  Q\(t')  d t’ . 

Jta 


(14) 


Thus,  the  stochastic  diffusion  process  i p(t,  to)  according  to  model  (13)  is  given  as  the 
sum  of  a  random  component,  linear  in  time,  plus  a  time-fluctuating  component 
proportional  to  the  diffusion  process  0(t,  to). 

The  objective  is  to  validate  the  model  in  Eq.  (11)  by  decomposing  the  damage 
increment  measure  \J/(t,  t0)  into  two  parts  that  are  mutually  statistically  independent 
and,  at  the  same  time,  equivalent  to  the  two  components  of  the  right-hand  side  of 
Eq.  (13).  That  is,  we  would  like  to  obtain  an  estimate  ij/(t,to)  of  the  stochastic 
damage  increment  measure  ij/(t,  to)  and  of  the  fluctuations  i J/(t,  to)  around  ij/(t,  to) 
from  the  initial  instant  to  to  the  current  instant  t  such  that 

i Kt,  to)  =  i }(t,  to)  +  i }{t,  to) ,  (15) 

where  t j/(t,  to)  is  statistically  equivalent  to  ASmQo(t  —  to),  and  t j/(t,  to)  is  statistically 
equivalent  to  (AS)'"0(t,to)  of  Eq.  (13). 

To  test  the  validity  of  the  above  postulate  that  the  two  components  of  the 
multiplicative  random  process  Qo(C)  and  Qi(C,  0  in  Eq,  (9)  are  statistically 
independent,  we  require  that  the  zero-mean  estimation  error  ij/(t,  to)  be  statistically 
orthogonal  to  the  estimate  of  the  increment  measure  t j/(t,  to)  in  the  Hilbert  space 
Li{P)  defined  by  the  probability  measure  P.  As  such  i }(t,  to)  is  the  best  linear  estimate 
of  the  stochastic  diffusion  process.  Based  on  mean-square  continuity  of  the  damage 
measure  t j/(t,  t0),  the  next  section  elaborates  on  the  model  structure  laid  out  in 
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Eq.  (15).  To  this  end,  we  analyze  experimental  data  sets  of  random  fatigue  via  KL 
decomposition  [38-40]  that  guarantees  the  above  statistical  orthogonality  among  the 
components  of  the  decomposition.  In  Section  4  we  also  use  these  experimental  data 
sets  to  identify  the  model  parameters. 


3.  Karhunen-Loeve  decomposition  of  experimental  data 

In  this  section  we  analyze  fatigue  test  data  via  KL  decomposition  [40]  to  justify  the 
model  structure  postulated  in  Eqs.  (11)  and  (12).  We  use  the  experimental  data  of 
random  fatigue  crack  growth  in  the  7075-T6  aluminum  alloy  [33]  and  conduct  the 
tests  under  different  constant  load  amplitudes  at  ambient  temperature.  For  all 
experiments  the  half-width  is  w  —  50.8  mm,  the  initial  crack  length  is  ato  =  9  mm, 
and,  therefore,  the  initial  dimensionless  crack  length  is  cto  =  a,0/w  =  0.18  with 
probability  1 .  The  Ghonem  data  sets  were  generated  for  60  center-cracked  specimens 
each  at  three  different  constant  load  amplitudes:  (i)  Set  #1  with  peak  nominal  stress 
of  70.65  MPa  (10.25  ksi)  and  stress  ratio  R  =  smm/SmiiX  =  0.6  for  54,000  cycles,  the 
effective  stress  range  A Se  =  15.84  MPa;  (ii)  Set  #2  with  peak  nominal  stress  of 
69.00  MPa  (10.00  ksi)  and  R  —  0.5  for  42,350  cycles,  and  A Se  =  17.80  MPa;  and  (iii) 
Set  #3  with  peak  nominal  stress  of  47.09  MPa  (6.83  ksi),  R  —  0.4  for  73,500  cycles, 
and  A Se  —  13.24  MPa.  The  three  experimental  data  sets  [33]  are  shown  in  the  three 
panels  of  Fig.  1. 

The  KL  decomposition  requires  the  mean  and  covariance  of  the  stochastic 
measure  of  damage  increment  i p(t,  t0 )  which  are  expressed  as 

to)  =  (i K*,  to))  , 

C^(tut2;  t0)  =  #(0,  t0)  -  ^(A,  to)][Ht2,  to)  ~  7V(?2’  r»)]>  •  (16) 

The  covariance  function  C^(t\ ,  t2,  to)  in  Eq.  (16)  is  continuous  at  t\  =  t2  =  t  for  all 
t^tQ.  Hence,  the  process  i p(t,  to)  is  mean-square  (ins)  continuous  based  on  a  standard 
theorem  of  mean-square  calculus  [38,39].  The  mean  and  covariance  are  calculate  for 
the  60  available  center-cracked  specimens  in  each  case. 

Since  only  finitely  many  data  points  at  n  discrete  instants  are  available  from 
experiments,  an  obvious  approach  to  the  analysis  of  the  damage  estimate  is  to 
discretize  over  the  finite  time  horizons  [?o,  t]  so  that  the  stochastic  process  i p(t,  to) 
becomes  the  ^-dimensional  random  vector  t//.  Consequently,  the  covariance  function 
C^(t\,t2;  to)  in  Eq.  (16)  is  reduced  to  a  real  semipositive-definite  ( n  x  n)  symmetric 
matrix  C^.  Since  the  experimental  data  were  collected  at  sufficiently  close  intervals, 
C ^  contains  pertinent  information  of  the  crack  damage  process.  The  n  (real  non¬ 
negative)  eigenvalues  of  are  ordered  as  X\  ^X2^  ■  ■  ■  5=1„,  with  the  corresponding 
eigenvectors,  i pl,cp2, . . .  ,cp",  which  form  an  orthogonal  basis  of  SJ1"  for  signal 
decomposition.  The  KL  decomposition  also  ensures  that  the  n  random  coefficients  of 
the  basis  vectors  are  statistically  orthogonal,  i.e.,  they  have  zero  mean  and  are 
mutually  uncorrelated.  These  random  coefficients  form  a  random  vector  X  = 
[x\,x2, . . .  ,xn\T  having  the  covariance  matrix  CXx  =  diag(z],Z2, . . .  ,A„),  leading  to  a 
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decomposition  of  the  discretized  signal  as 

•A =(«A>  +  X!  •  (17) 

j=  1 

It  was  observed  by  Ray  [37]  that  the  statistics  of  crack  length  are  dominated  by  the 
random  coefficient  corresponding  to  the  principal  eigenvector  (i.e.,  the  eigenvector 
associated  with  the  largest  eigenvalue)  and  that  the  combined  effects  of  the 
remaining  eigenvectors  are  small.  Therefore,  the  signal  i j/  in  Eq.  (17)  is  expressed  as 
the  sum  of  a  principal  part  and  a  (zero-mean)  residual  part  that  are  mutually 
statistically  orthogonal: 

/ 

i A  =  (i A)  +  x\ cj)1  +  ^2  x$  ■  (18) 

principal  part  y_ 2 

residual  part 

Thus,  as  Eq.  (15)  requires,  the  vector  \jj  is  expressed  as  the  sum  of  the  principal  and 
residual  parts  with  equality  in  the  mean  square  (ms)  as 

«A  =  <A  +  <A,  (19) 

where  the  principal  part  is  the  damage  estimate 

< }  =  {< A)  +x\ 01  ,  (20) 

the  residual  part  is  the  estimation  error  representing  the  fluctuations  around  the 
mean  damage  estimate  (20) 

^  =  ’  (21) 
7=2 

and  the  resulting  (normalized)  mean  square  error  [40]  is 

2  =  Trace{Cov[il/  -  i }]}  =  E/'=227 

Enm  Trace{Cov[i) (]}  EyLi^ 

The  KL  decomposition  of  fatigue  test  data  sets  reveals  that  0.01  ^£"mj ^0.1  for  all 
three  data  sets. 

The  principal  eigenvector  cf)l(t),  associated  with  the  largest  eigenvalue  X\,  closely 
fits  the  ramp  function  ( t  —  to)  for  each  of  the  three  data  sets  in  Fig.  1;  this  is  shown  in 
Fig.  2  for  the  data  set  1 .  Comparing  the  terms  on  the  right-hand  side  of  the  discrete 
model  in  Eq.  (19)  with  those  of  the  continuous  model  in  Eq.  (13),  it  is  reasonable  to 
have  the  random  variable  AS'”[£>o  —  /-Iq]  equal  (in  ms  sense)  to  the  random  coefficient 
x i  of  the  principal  eigenvector  cpl(t).  Applying  the  lemma  from  the  appendix,  a 
mean-square  equivalence  between  the  KL  decomposition  model  in  Eq.  (19)  derived 
from  the  test  data  and  the  postulated  model  in  Eq.  (17)  is  established  as 

ms 

m))  »  \Smtio(t-to)  ,  (23) 

discrete  model  (test  data)  continuous  model  (constitutive  relation) 
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(a)  Time  in  units  of  50  cycles  (1  cycle  =  100  ms) 


Fig.  2.  (a)  Curves  t //(t)  obtained  with  Eq.  (12)  for  the  experimental  data  of  7075-T6  aluminum  alloy  for  set 
#1 .  The  value  of  m  used  is  m  =  4.0.  (b)  Principal  part  of  the  KL  decomposition  against  (c)  the  linear 
approximation  of  the  continuous  model  made  in  Eqs.  (23)  plus  (24)  of  the  curves 
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*i 4>\i)  ™  ASm[Q0  ~  )k>](t  ~  to)  ,  (24) 

discrete  model  (test  data)  continuous  model  (constitutive  relation) 

n 

Yl,x$  “  (AS)m0(t,  tQ)  .  (25) 

2  continuous  model  (constitutive  relation) 

discrete  model  (test  data) 

Thus,  we  have  i ]/  =  (i/d?)}  +  xi  <pl(t)  ~  ASmQo(t  —  hi),  and  i ]/  =  YTj=ix$  ~ 
AS"'0(t,to)  as  assumed  in  Eq.  (15).  The  two  entities  on  the  left-hand  side  in  Eqs. 
(24)  and  (25)  are  mutually  statistically  orthogonal  by  construction.  Similarly,  in  view 
of  Eq.  (15),  the  zero-mean  estimation  error  \j/(t,  to)  is  statistically  orthogonal  to 
i j/(t,  to)  in  the  Hilbert  space  Li{P)  defined  by  the  probability  measure  P  associated 
with  the  stochastic  process  \{/(t,  to).  As  such  ip(t,to)  can  be  viewed  as  the  best  linear 
estimate  of  i j/(t,  t0)  with  the  least  error  \j/(t,  t0)  in  the  mean-square  sense. 

It  follows  from  Eqs.  (15)  to  (25)  that  the  uncertainties  associated  with  an 
individual  sample  resulting  from  the  damage  measure  estimate  ij/(t,  to)  dominate  the 
cumulative  effects  of  material  inhomogeneity  and  measurement  noise  in  the 
estimation  error  i j/(t,  to)  unless  ( t  —  to)  is  small.  Therefore,  from  the  perspectives  of 
material-health  monitoring,  risk  analysis,  and  remaining  life  prediction  where  the 
inter-maintenance  interval  ( t  —  to)  is  expected  to  be  large,  a  reasonably  accurate 
identification  of  the  mean  n0  and  variance  a\  of  the  random  parameter  Qq  is  crucial, 
while  the  role  of  the  diffusion  process  0(t,  to)  is  relatively  less  significant.  This 
observation  is  consistent  with  the  statistical  analysis  of  fatigue  test  data  by  Ditlevsen 
[2],  where  the  random  process  described  by  Eq.  (25)  is  treated  as  the  zero-mean 
residual.  Ditlevsen  [2]  also  observed  largely  similar  properties  by  statistical  analysis. 
Nevertheless,  the  stochastic  properties  of  fluctuating  function  0 (t,  to),  which  we 
investigate,  can  disclose  important  information  about  the  material  structure  of  alloys 
during  crack  damage. 


4.  Data  analysis 

In  this  section  we  investigate  the  stochastic  equivalence  made  in  Eq.  (25)  between 
the  residual  component  of  the  signal  as  obtained  by  the  KL  decomposition  and  the 
linear  approximation.  The  first  step  is  to  evaluate  the  exponent  parameter  m  by 
fitting  the  data  of  the  crack  growth  with  Eq.  (8).  The  fit  is  done  by  considering  the 
crack  increments  from  all  60  cases  for  each  of  the  three  experiments. 

By  using  the  empirical  values  of  m  it  is  possible  to  estimate  \ji(t,to)  via  Eq.  (12). 
The  three  plots  in  Fig.  2  compare  the  curve  i jj(t,  to),  its  principal  part  according  to  the 
KL  decomposition  and  its  linear  approximation  according  to  the  continuous  model 
made  in  Eqs.  (23)  plus  (24)  for  set  #1:  the  figures  for  the  other  data  sets  look 
qualitatively  similar.  Fig.  3  shows  the  quality  of  the  equivalence  made  in  Eqs.  (23) 
plus  (24)  between  the  discrete  model,  which  makes  use  of  the  KL  decomposition,  and 
the  continuous  model,  which  makes  use  of  a  linear  approximation. 
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Fig.  3.  Increments  Sc,  against  crack  length  c,  fit  with  Eq.  (8)  (solid  curve)  for  the  experimental  data  of 
7075-T6  aluminum  alloy  for  the  set  #1:  QASm  =  0.0019  ±  0.0002  and  m  =  4.0  ±  0.2. 


Fig.  3  shows  the  fitted  data  and  the  results  for  set  #1;  the  figures  for  the  other  sets 
are  similar.  The  parameters  for  all  the  three  sets  are  listed  below. 

•  Q(C)AS'n  =  0.0019  ±  0.0002  and  m  =  4.0  ±  0.2  for  set  #1; 

•  Q(C)AS'"  =  0.0022  ±  0.0002  and  m  =  3.8  ±  0.2  for  set  #2; 

•  Q(C)ASm  =  0.0018  ±  0.0002  and  m  =  4.7  ±  0.2  for  set  #3. 


4.1.  Diffusion  standard  deviation  analysis  of  the  fluctuations 

We  evaluate  the  stochastic  equivalence  made  in  Eq.  (25)  between  the  residual  part 
of  the  discrete  model,  which  makes  use  of  the  KL  decomposition,  and  the  residual 
part  of  the  continuous  model,  which  makes  use  of  a  linear  approximation,  in  two 
steps.  Step  1  compares  the  size  of  the  increments  of  the  correspondent  residual  parts; 
and  Step  2  adopts  the  standard  deviation  analysis  (SDA)  which  is  a  statistical 
formalism  to  study  the  long-time  correlation  in  a  fractal  time  series. 

Because  0(t)  =  residual  part ,  the  increments  are  given  by  9,  =  0(t)  —  0(t  —  1). 
We  calculate  the  standard  deviation,  og,  of  the  increments  {0r}  for  each  residual 
component  estimated  by  means  of  the  KL  decomposition  and  of  the  linear 
approximation,  respectively.  Finally,  we  calculate  the  average  of  the  standard 
deviation,  (a g),  between  the  60  og  for  each  of  the  three  cases.  The  results  shown  in 
Table  1  demonstrate  the  compatibility  of  the  increments  obtained  with  the  residual 
parts  of  the  KL  decomposition  and  the  residual  part  of  the  continuous  model. 

Now,  let  us  suppose  that  a  generic  residual  curve  is  given  by  the  function  <9(0,  see 
Eq.  (25),  that  in  this  specific  case  is  a  kind  of  random  walk  around  the  ballistic  part 


14 


N.  Scafetta  et  al.  /  Pliysica  A  359  (2006)  1-23 


Table  1 

Values  of  the  fitting  parameters  fi  and  a  of  the  lognormal  distribution  (29)  of  the  histograms  shown  in 
Fig.  7 


o 

Set  #1 

0.58  ±0.05 

0.20  ±0.02 

Set  #2 

0.74  ±0.05 

0.16  ±0.02 

Set  #3 

0.42  ±  0.05 

0.45  ±  0.04 

of  the  signal,  which  is  the  principal  component  of  the  KL  decomposition  or  the 
linear  component  of  the  continuous  model.  The  SDA  determines  the  scaling  of  the 
standard  deviation  of  the  diffusion  process  defined  as 


D(  t)  = 


[0(t  +  T)  -  0(0  -  0{t  +  t)  -  0(O]2 
N- T-  1 


(26) 


where 


0(t  +  t)  —  0(0  = 


N—t 


E 


0(t  +  r)  —  0(0 
N-r 


(27) 


N  is  the  number  of  data  points  and  times  t  and  t  are  measured  in  cycle  period  units. 
It  is  easy  to  realize  that  Eq.  (27)  ensures  that  D( x  =  1)  =  1.  In  the  presence  of 
fractal  statistics  we  would  have,  based  on  the  discussion  of  anomalous  diffusion  in 
Section  1, 

D( t)  oc  /  =  ra/2  .  (28) 

Fig.  4  shows  the  SDA  for  the  residual  part  of  the  KL  decomposition.  Each  set  of 
graphs  concerning  the  same  crack  data  look  quite  similar.  All  three  sets  of  graphs 
show  that  the  curves  have  a  initial  scaling  exponent  approximately  within  the  range 
0.5  < /l <0.9.  The  mean  curve  value  is  represented  by  the  curves  with  black  circles  in 
Fig.  4.  These  early  time  values  of  /?,  interpreted  in  terms  of  the  random  walks 
discussed  in  Section  1,  indicate  that  the  residual  parts  of  the  signal  manifest  a 
persistent  behavior,  i.e.,  a  persistent  correlation  that  lasts  at  least  10  consecutive 
cycles  on  average. 

For  1 0 < t <  100  the  data  present  a  slight  antipersistency  with  0.4<7F<0.5. 
Consequently,  the  residual  process  is  initially  strongly  persistent,  but  asymptotically 
it  is  almost  random.  We  observe  that  for  10<t<  100  the  mean  scaling  exponent  is 
approximately  H  =  0.45  in  the  case  of  the  linear  continuous  model  and  this  is 
slightly  larger  than  the  scaling  exponent  in  the  KL  discrete  decomposition.  This 
change  in  scaling  is  due  to  the  fact  that  the  principal  part  obtained  with  the  KL 
decomposition  extracts  more  information  from  the  original  signal  than  does  the 
simple  linear  approximation. 
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Fig.  4.  SDA  for  the  residual  part  of  the  KL  decomposition  (left  figures)  against  SDA  for  the  residual  part 
of  linear  approximation  (right  figures)  of  the  continuous  model.  Note  the  scaling  transition  ati  s:  10  from 
H  ~  0.7  to  H  ~  0.5  in  both  cases  for  all  data  sets. 


In  Section  1  we  have  explained  that  an  initial  anomalous  diffusion  that  lasts  up  to 
a  certain  t  as  detected  by  Eq.  (1)  could  also  be  an  artifact  related  not  to  some 
autocorrelation  pattern  in  the  data  but  to  the  transition  front  the  initial  geometrical 
properties  of  the  distribution  of  the  events  {£,•}  of  a  time  series  to  the  Gaussian  shape 
of  the  asymptotic  diffusion  distribution.  To  check  that  the  persistent  behavior  for 
t<  10  observed  in  the  plots  of  Fig.  4  expresses  real  correlation  patterns,  we  repeat 
SDA  of  the  data  after  randomizing  the  time  series  of  the  increments  {9t}.  That  is,  for 
each  crack  data  first  we  have  the  sequence  {0f}  defined  as  9,  —  0(t )  —  0(t  —  1),  then 
we  shuffle  {0,}  and  obtain  a  new  sequence  {0'}  and  generate  a  new  walk 
0'(t)  =  Y?i=  i  Q'n  and  finally  we  apply  SDA  to  the  new  curve  0'{t).  Fig.  5  shows 
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Fig.  5.  SDA  for  the  residual  part  of  the  KL  decomposition  after  shuffling  of  the  increments  {0,}.  Note  the 
random  scaling  of  H  ^  0.5.  The  data  refer  to  set  #1  and  the  comparison  has  to  be  made  with  Fig.  4  (crack 
data  1)  KL. 


the  result  for  the  crack  set  #1  where  the  residual  part  is  estimated  with  the  KL 
decomposition;  for  the  other  data  sets  the  results  are  similar.  Fig.  5  clearly  shows 
that  after  shuffling  of  the  temporal  order  of  the  single  increments  {9t},  the  SDA  of 
the  new  sequence  gives  a  scaling  value  of  approximately  H  =  0.5  and  the  persistent 
behavior  fort  <  10  observed  in  Fig.  4  is  absent.  Thus,  we  conclude  that  the  persistent 
behavior  for  t<10  observed  in  Fig.  4  expresses  real  correlation  patterns  in  the 
fluctuations  of  crack  growth. 

Fig.  6  also  shows  that  the  distributions  of  the  scaling  exponent  seems  to  be  quite 
uniform  in  the  interval  0.5 < /? < 0.9  (with  a  probability  P> 0.9%)  or,  perhaps,  as 
Fig.  6c  shows  better,  there  might  be  a  slight  prominence  or  skewness  in  favor  of  small 
value  of  /?.  In  any  case,  all  figures  show  that  the  distribution  of  the  scaling  exponent 
for  the  residual  components  of  the  curve  obtained  with  the  KL  decomposition  or  the 
linear  component  of  the  continuous  model  practically  coincide  for  all  three  data  sets. 
This  equivalence  suggests  that  the  continuous  linear  model  essentially  captures 
not  only  the  dominant  properties  of  the  signal,  as  obtained  through  the  KL 
decomposition,  see  Eq.  (24),  but  also  the  stochastic  properties  of  the  residual  signal, 
as  suggested  in  Eq.  (25). 

4.2.  Statistics  of  damage  measure  estimates 

We  investigate  the  statistics  of  the  damage  measure  estimates  using  a  lognormal 
distribution.  This  is  in  keeping  with  the  analysis  of  several  investigators  who 
assumed  the  crack  growth  rate  in  ductile  alloys  is  lognormal-distributed  (see,  for 
example,  the  citations  in  Sobczyk  and  Spencer  [9]).  Other  investigators  have  treated 
the  crack  length  as  being  lognormal-distributed  [37],  rather  than  the  residual 
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(d)  scaling  exponent  - ft  - 


Fig.  6.  Flistogram  and  probability  density  function  of  the  scaling  exponent  /t  estimated  by  fitting  the  range 
1  <r<  =  10  for  each  curve  shown  in  the  plates  of  Fig.  4.  Each  plate  compares  the  distributions  obtained 
with  the  KL  decomposition  and  the  continuous  linear  model  for  each  of  the  three  crack  data  sets.  Fig.  7d 
shows  the  histograms  of  all  data. 


fluctuations.  The  results  of  KL  decomposition  in  Eqs.  (16)— (19)  are  consistent  with 
these  assumptions  because  O0.  which  dominates  the  random  behavior  of  fatigue 
crack  growth,  can  be  considered  to  be  a  perfectly  correlated  (ballistic)  random 
process,  whereas  the  non-negative,  multiplicative  uncertainty  term  0(t,  to)  is  a 
weakly  (positively)  correlated  random  process.  Yang  and  Manning  [39]  have 
presented  an  empirical  second-order  approximation  to  crack  growth  by  postulating  a 
lognormal  distribution  of  a  parameter  that  does  not  bear  any  physical  relationship  to 
AS  but  is,  to  some  extent,  similar  to  Qq(AS)  in  the  present  model. 

Fig.  7  shows  the  histogram  of  the  slopes  AS'"Qo  of  the  curves  according  to  the 
continuous  model  for  the  experimental  data  presented  by  Eq.  (25),  such  as  those 
shown  in  Fig.  3c.  The  histograms  are  fitted  with  the  lognormal  distribution  p(x,  fi,  cr): 


p(x;  fi,  c ) 


1 

xV  2no2 


exp 


(ln(x)  -  fi)2' 
2  o2 


(29) 


The  measured  parameters  fi  and  a  are  recorded  in  Table  2.  Finally,  the  parameters  )i 
and  a  are  functions  of  )i0  —  (x)  and  <75  =  ((x  —  )i0)2)  as  follows: 

p,  =  ln(/z0)  -  a1/ 2 


(30) 
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Fig.  7.  Histogram  of  the  quantities  AS’"Qo  of  the  continuous  model  of  the  experimental  data  presented  by 
Eq.  (25).  The  histograms  are  fitted  with  a  lognormal  distribution  p(x;  fi,  a)  shown  in  Eq.  (29). 
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Table  2 

Mean  standard  deviation  of  the  increments  of  the  residual  part  obtained  with  the  KL  decomposition  and 
the  residual  part  of  the  continuous  model 


KL 

Linear  model 

Set  #1:  (<7fl)  = 

0.0024  ±0.001 

0.0025  ±0.001 

Set  #2:  (<7fl)  = 

0.0024  ±0.001 

0.0025  ±0.001 

Set  #3:  (<7fl)  = 

0.0038  ±  0.003 

0.0043  ±  0.003 

and 


2  _ 


=  In 


1 


(31) 


Since  the  random  parameter  A S'”Qo  is  not  explicitly  dependent  on  time,  its  expected 
value  is  obtained  from  Eq.  (13)  as 

o)N 


Ho  =  (AS'"Qq)  = 


t- t0 


(32) 


which  is  readily  determined  from  the  ensemble  average  estimate  from  each  of  the 
data  sets.  Asymptotically  in  time  we  find  for  the  variance  of  AS'"Qo 


al  =  ((A S'”Q0  -  /t0)2} 


W,  to) 


t-  to 


2 

/i0 


(33) 


so  that  the  variance  can  be  determined  directly  from  the  ensemble  average  estimate 
from  each  of  the  data  sets. 


5.  Crack  model  simulation 

This  section  presents  the  results  of  Monte  Carlo  simulation  of  the  fatigue  crack 
damage  process  based  on  the  model  as  it  emerges  from  the  stochastic  analysis  made 
in  the  previous  section.  The  model  that  we  introduce  approximately  reproduces  the 
stochastic  properties  of  both  the  ballistic  or  principal  part  of  the  fatigue  crack 
growth  and  the  associated  fluctuations  around  it.  The  model  consists  in  generating 
independently  the  fluctuation  and  the  principal  part  of  the  fatigue  crack  damage  in 
such  a  way  that  they  are  statistically  equivalent  to  the  correspondent  observations 
and  then  combining  them.  The  crack  model  simulation  is  based  on  four  steps: 

•  Principal  part  or  ballistic  growth:  We  generate  60  values  of  AS'"Qo,  lognormally 
distributed  according  to  Eq.  (29),  where  the  parameters  /t  and  a  are  given  by  the 
actual  fit  of  the  phenomenological  distribution  shown  in  Fig.  6  and  recorded  in 
Table  2.  A  sample  of  the  curves  AS'”Qo(t  —  to)  simulating  the  data  set  #1  is  shown 
in  Fig.  8b. 

•  Residual  part  or  fluctuations  around  the  ballistic  growth:  We  generate  60  fractal 
Gaussian  noise  sequences  {0'}  each  of  length  N  of  the  original  time  sequence  and 
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(d)  Time  in  units  of  50  cycles  (1  cycle=1 00  ms) 


Fig.  8.  Synthetic  data  of  crack  length  for  set  #1 .  (a)  SDA  of  the  residual  component;  (b)  ballistic  growth; 
(c)  damage  increment  i (d)  crack  length,  compare  with  Fig.  la. 


with  scaling  exponent  uniformly  distributed  in  the  interval  0.5  < /I  <0.9.  The 
standard  deviation  of  each  sequence  is  set  equal  to  the  mean  standard  deviation  of 
the  increments  of  the  residual  component  of  the  data  reported  in  Table  1.  To 
simulate  the  change  of  scaling  exponent  from  persistent  (for  t<10)  to  almost 
random  (for  r>  10),  we  section  each  fractal  time  series  {$'}  into  segments  of  length 
10  within  which  the  data  would  conserve  the  correlation,  and  finally  we  shuffle  the 
position  of  these  segments  in  the  time  series  to  reproduce  a  new  time  series  {$,}. 
These  new  time  series  will  have  persistent  correlation  for  t<  10  and  uncorrelated 
randomness  for  t>  10.  Finally,  the  curve  &(t)  is  obtained  by  integrating  the  new 
sequence  {6,}  and  by  detrending  from  it  its  linear  component  because  the  curve 
©(f)  is  supposed  to  have  a  zero  mean.  The  SDA  sample  data  analysis  of  an 
example  of  these  synthetic  residual  data  simulating  the  data  set  #1  is  shown  in 
Fig.  8a. 

•  The  ballistic  growth  estimated  in  the  principal  part  and  the  associated  fluctuations 
of  the  residual  part  are  combined  according  to  Eq.  (18)  to  obtain  a  simulated 
damage  increment  measure  i <J/(t,  to)  for  all  60  sequences  and  for  the  three  data  sets. 
Fig.  8c  shows  the  simulated  damage  increment  measure  i/df,  to)  simulating  the 
data  set  #1. 
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•  Finally,  by  using  the  respective  value  of  the  exponent  m,  reported  in  Section  4,  for 
the  data  set  #1  and  a  one-dimensional  root-finding  computer  algorithm,  Eq.  (12) 
is  inverted  to  obtain  a  simulated  normalized  crack  length  growth  curves  ct,  as  seen 
in  Fig.  8d.  The  similitude  between  Figs.  8d  and  la  is  noteworthy  and  the  figures 
for  the  other  data  sets  look  qualitatively  very  similar;  hence  they  are  not  presented 
in  this  paper. 


6.  Summary  and  conclusions 

This  paper  presents  a  stochastic  measure  of  fatigue  crack  damage.  We  have 
focused  on  the  correlation  properties  of  the  fluctuations  around  fatigue  crack 
growth  in  ductile  alloys.  The  model  of  crack  damage  measure  indicates  that  the 
fluctuations  around  fatigue  crack  growth  present  strong  correlation  patterns  within 
short-time  scales  and  are  uncorrelated  for  larger  time  scales.  These  Endings  suggest 
that  the  random  stochastic  models  adopted  in  the  present  literature  for  describing 
the  crack  growth  dynamics  should  be  augmented  with  short-time  correlated 
stochastic  models. 

The  damage  measure  is  modeled  as  an  anomalous  diffusion  process  that  is 
obtained  as  a  continuous  function  of  the  current  crack  length  and  of  the  initial  crack 
length.  Perhaps,  the  randomness  in  the  damage  measure  estimate  accrues  primarily 
from  manufacturing  uncertainties  such  as  defects  generated  during  machining 
operations  because  such  macro-defects  are  expected  to  drive  the  ballistic  growth  of 
cracks.  This  randomness  is  captured  by  a  single  lognormal-distributed  random 
variable.  Instead,  the  resulting  diffusion  process  of  estimated  fluctuations  around 
the  ballistic  growth  of  fatigue  cracks  is  probably  due  to  the  inhomogeneity  in  the 
structural  material  because  it  is  primarily  associated  with  the  micro-structure  of 
the  material,  and  is  represented  by  a  non-stationary  fractional  Brownian  motion 
model.  This  non-stationarity  manifests  itself  in  the  two  scaling  exponents  occ¬ 
urring  at  different  scales.  Specifically,  we  observe  a  clear  transition  in  the  stan¬ 
dard  deviation  analysis  from  an  early  time  slope  representing  a  strong  persistence, 
/?  ~  0.7  lasting  for  approximately  i  ^  10  to  a  different  slope  asymptotically  in  time 
representing  randomness,  ft  ~  0.5.  This  transition  occurring  at  r  ^  10  from  a 
scaling  regime  to  another  indicates  the  scale  at  which  a  structure  change  of  the 
ductile  alloys  occurs. 

The  constitutive  equation  of  the  damage  measure  is  based  on  the  physics  of 
fracture  mechanics  and  is  validated  by  KL  decomposition  of  fatigue  test  data  for 
7075-T6  aluminum  alloys  at  different  levels  of  (constant-amplitude)  cyclic  load.  The 
damage  estimate  is  statistically  orthogonal  to  the  resulting  zero-mean  estimation 
error  in  the  Hilbert  space  LiiP)  defined  by  the  probability  measure  of  the  stochastic 
damage  measure.  As  such,  the  damage  estimate  is  often  viewed  as  a  best  least-square 
linear  estimate.  However,  we  find  that  the  KL  decomposition  is  statistically 
equivalent  to  the  linear  approximation  in  the  continuum  model  that  can  be  then  used 
to  simulate  the  fatigue  crack  growth  in  ductile  alloys. 
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Appendix:  A  supporting  lemma 

Lemma.  Let  A(f)  and  B(Q  be  second-order  real  random  variables',  x(£,  t)  and  v(£,  t )  be 
zero-mean  mean-square  continuous  (possibly  non-separable )  real  random  processes', 
and  the  reed  g(t)  be  almost  everywhere  continuous  on  an  interval  A  such  that,  for  all 
t  e  A,  the  following  conditions  hold'. 

(i)  A(C)  =  B(0; 

(ii)  (ACC)x(f  i))  =  0  and  (B(f)y(f  t))  -  0. 

Then,  the  following  mean-square  identity. 

A(C)g(t)  +  x(L  t )  =  B(0g(t )  +  y(L  t) 

yields 

x(C,  t)  =  y(C,  t)  \ 

{A(f)y(L  t))  =  0  l  \/teA. 

(B(f)x(fty)  =  0  J 

Proof.  It  follows  from  the  above  mean-square  identity  that 
Var[{A{ 0  -  B(Q)g(t)  +  {x(£,  t)  -  y(f  t))}  =  0 
which  may  be  expanded  to  yield 

Var[A( 0  -  B(0]g(tf  +  Var[x(f  t )  -  y(C,  0] 

+  <M(0  -  BOWxC.  t)  -  y(C,  t)})g{t)  -  0  . 

A  combination  of  condition  (i)  and  Schwarz  inequality  yields 

Var[x(f,t)-y(U)\  =  0 

and  the  remaining  two  identities  follow  from  condition  (ii).  □ 
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This  paper  reviews,  expands,  and  clarifies  the  underlying  concepts  of  a  signed  real  measure  of 
regular  languages,  which  has  been  used  as  a  novel  tool  for  synthesis  of  discrete  event 
supervisory  control  systems.  The  language  measure  is  constructed  upon  the  principles  of 
automata  theory  and  real  analysis.  It  allows  total  ordering  of  a  set  of  partially  ordered 
sublanguages  of  a  regular  language  for  quantitative  evaluation  of  the  supervised  behaviour 
of  deterministic  finite  state  automata  (DFSA)  under  different  supervisors.  In  the  setting  of 
the  language  measure,  a  supervisor’s  performance  is  superior  if  the  supervised  plant  is  more 
likely  to  terminate  at  a  good  marked  state  and/or  less  likely  to  terminate  at  a  bad  marked 
state.  The  computational  complexity  of  the  language  measure  algorithm  is  polynomial  in 
the  number  of  DFSA  states. 


1.  Introduction 

Discrete  event  systems  belong  to  a  special  class  of 
dynamical  systems.  The  states  of  a  discrete  event 
system  may  take  discrete  (or  symbolic)  values  and 
change  only  at  (possibly  asynchronous)  discrete  instants 
of  time,  in  contrast  to  the  familiar  continuously  varying 
dynamical  systems  of  the  physical  world,  which  can  be 
modelled  by  differential  or  difference  equations.  The 
dynamics  of  many  human-engineered  systems  evolve 
asynchronously  in  time  via  complex  interactions  of 
various  discrete-valued  events  with  continuously  varying 
physical  processes.  The  relatively  young  discipline  of 
discrete  event  systems  has  undergone  rapid  growth 
over  the  last  three  decades  with  the  evolution  of 
human  engineered  complex  systems,  such  as  integrated 
control  and  communication  systems,  distributed  sensing 
and  monitoring  of  large-scale  engineering  systems, 
manufacturing  and  production  systems,  software 
fault  management,  and  military  Command,  Control, 
Computer,  Communication,  Intelligence,  Surveillance, 
and  Reconnaissance  ( C4ISR )  systems. 

The  discipline  of  discrete  event  systems  was  initiated 
with  simulation  of  human-engineered  processes  about 


four  decades  ago  in  the  middle  of  nineteen  sixties.  The 
art  of  discrete  event  simulation  emerged  with  the  devel¬ 
opment  of  a  simulation  software  package,  called  GPSS, 
that  was  followed  by  numerous  other  software  simula¬ 
tion  tools,  such  as  SIMSCRIPT  II. 5,  SLAM  II,  and 
SIMAN  (Law  and  Kelton  1991).  Shortly  thereafter, 
computer  scientists  and  control  theorists  entered  the 
field  and  brought  in  theoretical  concepts  of  languages 
and  automata  in  modelling  discrete  event  systems.  In 
the  late  nineteen  sixties,  Arbib  (see  Kalman  et  al.  1969) 
showed  how  algebraic  methods  could  be  used  to  explore 
the  structure  of  finite  automata  to  model  dynamical 
systems.  Around  that  time,  computer  scientists  focused 
on  formal  languages,  automata  theory,  and  computa¬ 
tional  complexity  for  application  of  language-theoretic 
concepts  (e.g.,  regular  expressions  and  context-free 
grammars)  in  software  development  including  design  of 
compilers  and  text  processors  (Yu  1997,  Hopcroft  et  al. 
2001).  In  the  late  nineteen  seventies  and  early  nineteen 
eighties.  Ho  and  co-workers  introduced  the  concept  of 
finite  perturbation  in  discrete  event  systems  for 
modelling  and  analysis  of  human-engineered  systems 
(Ho  and  Cao  1991).  So  far,  no  concrete  theoretical 
concept  and  mathematical  tools  had  been  available  for 
analysis  and  synthesis  of  discrete  event  control  systems. 

The  concept  of  discrete  event  supervisory  (DES) 
control  was  first  introduced  in  the  seminal  paper  of 
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Ramadge  and  Wonham  (1987)  and  this  important 
paradigm  has  been  subsequently  extended  by  other 
researchers  (for  example,  see  citations  in  Kumar  and 
Garg  (1995)  and  Cassandras  and  Lafortune  (1999), 
and  the  October  2000  issue  of  Part  B  of  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics). 
These  efforts  have  led  to  the  evolution  of  a  new 
discipline  in  decision  and  Control,  called  Supervisory 
Control  Theory  (SCT),  that  requires  partitioning  the 
discrete-event  behaviour  of  a  physical  process,  called 
the  plant,  into  legal  and  illegal  categories.  The  legal 
behaviour  of  plant  dynamics  is  modelled  by  a  determi¬ 
nistic  finite-state  automaton,  abbreviated  as  DFSA  in 
the  sequel.  The  DFSA  model  is  equivalent  to  a  regular 
language  that  is  built  upon  an  alphabet  of  finitely 
many  events;  the  event  alphabet  is  partitioned  into 
subsets  of  controllable  events  (that  can  be  disabled) 
and  uncontrollable  events  (that  cannot  be  disabled). 
Based  on  the  regular  language  of  an  unsupervised 
plant,  SCT  synthesizes  a  DES  controller  as  another 
regular  language,  having  the  common  alphabet  with 
the  plant  language,  that  guarantees  restricted  legal 
behaviour  of  the  supervised  plant  based  on  the  desired 
specifications.  Instead  of  continuously  handling  numeri¬ 
cal  data,  DES  controllers  are  designed  to  process  event 
strings  to  disable  certain  controllable  events  in  the 
physical  plant.  A  number  of  algorithms  for  DES  control 
synthesis  have  evolved  based  on  the  automata  theory 
and  formal  languages  relying  on  the  disciplines  of 
Computer  Science  and  Control  Science.  In  general,  a 
supervised  plant  DFSA  is  synthesized  as  a  parallel 
composition  of  the  unsupervised  plant  DFSA  and  a 
supervisor  DFSA  (Cassandras  and  Lafortune  1999). 
The  supervised  plant  DFSA  yields  a  sublanguage 
of  the  unsupervised  plant  language,  which  enables 
restricted  legal  behaviour  of  the  supervised  plant 
(Ramadge  and  Wonham  1987,  Kumar  and  Garg  1995, 
Cassandras  and  Lafortune  1999).  These  concepts  have 
been  extended  to  several  practical  applications,  includ¬ 
ing  hierarchical  Command,  Control,  Communication, 
and  Intelligence  (C3/)  systems  (Phoha  et  al.  2002). 
Apparently,  there  have  been  no  quantitative  methods 
for  evaluating  the  performance  of  supervisory  control¬ 
lers  and  establishing  thresholds  for  their  performance. 

The  concept  of  permissiveness  has  been  used  in  DES 
control  literature  (Kumar  and  Garg  1995,  Cassandras 
and  Lafortune  1999)  to  facilitate  qualitative  comparison 
of  DES  controllers  under  the  language  controllability 
condition.  Design  of  maximally  permissive  DES 
controllers  has  been  proposed  by  several  researchers 
based  on  different  assumptions.  However,  maximal 
permissiveness  does  not  imply  best  performance  of  the 
supervised  plant  from  the  perspective  of  achieving 
plant  operational  objectives.  For  example,  in  the 
travelling  salesman  problem,  a  maximally  permissive 


supervisor  may  not  yield  the  least  expensive  way  of 
visiting  the  scheduled  cities  and  returning  to  the  starting 
point  because  no  quantitative  measure  of  performance 
is  addressed  in  this  type  of  supervisor  design. 

The  above  argument  evinces  the  need  for  a  signed  real 
measure  of  regular  languages,  which  can  be  used  for 
quantitative  evaluation  and  comparison  of  different 
supervisors  for  a  physical  plant,  instead  of  relying  on 
permissiveness  as  the  (qualitative)  performance  index. 
Construction  of  the  proposed  language  measure  follows 
Myhill-Nerode  Theorem  (Martin  1997,  Hopcroft  et  al. 
2001),  which  states  that  a  regular  language  can  be 
partitioned  into  finitely  many  right-invariant  equiva¬ 
lence  classes.  In  other  words,  a  state-based  partitioning 
of  the  (unsupervised)  plant  language  yields  equivalence 
classes  of  finite-length  event  strings.  Each  marked  state 
is  characterized  by  a  signed  real  value  that  is  chosen 
based  on  the  designer’s  perception  of  the  state’s 
impact  on  the  system  performance.  Conceptually  similar 
to  conditional  probability,  each  event  is  assigned  a  cost 
based  on  the  state  at  which  it  is  generated.  This 
procedure  permits  a  string  of  events,  terminating  on  a 
good  (bad)  marked  state,  to  have  a  positive  (negative) 
measure.  A  supervisor  can  be  designed  in  this  setting 
such  that  the  supervisor  attempts  to  eliminate  as  many 
bad  strings  as  possible  and  retain  as  many  good  strings 
as  possible.  Different  supervisors  may  achieve  this  goal 
in  different  ways  and  generate  a  partially  ordered  set 
of  supervised  sublanguages.  The  language  measure 
then  creates  a  total  ordering  on  the  performance  of 
the  supervised  sublanguages,  which  provides  a  precise 
quantitative  comparison  of  the  controlled  plant 
behaviour  under  different  supervisors.  This  feature  is 
formally  stated  as  follows. 

Given  that  the  relation  C  induces  a  partial  ordering  on  a  set 
of  supervised  sublanguages  {L(Sj /G),  j  =  1,  ...,1V}  of  the 
plant  language  L(G )  under  supervisors  whose  languages 
are  [L{S'),j  =  the  language  measure  p.  induces 

a  total  ordering  <  on  {p.{L{Sj /G))}.  In  other  words ,  the 
range  of  the  set  function  gt  is  totally  ordered  while  its 
domain  could  be  partially  ordered. 

The  above  problem  was  first  addressed  by  Wang  and 
Ray  (2004)  who  proposed  a  signed  measure  of  regular 
languages;  an  alternative  approach  was  proposed  by 
Ray  and  Phoha  (2003)  who  constructed  a  vector  space 
of  formal  languages  and  defined  a  metric  based  on  the 
total  variation  measure  of  the  language. 

This  paper  reviews,  clarifies  and  expands  the  contents 
of  previous  publications  (Ray  and  Phoha  2003,  Wang 
and  Ray  2004)  from  the  perspectives  of  discrete-event 
supervisory  control  within  a  unified  framework  and 
also  introduces  new  concepts  and  ramifications  of  the 
language  measure  and  its  parameter  identification. 
Systematic  procedures  for  computation  of  the  language 
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measure  are  developed  in  this  paper  and  they  are 
illustrated  with  an  engineering  example.  The  major 
objective  here  is  rigorous  formulation  and  systematic 
construction  of  a  real  signed  measure  of  regular 
languages,  based  on  the  fundamental  principles  of 
automata  theory  and  real  analysis.  The  quantitative 
tools  are  readily  applicable  to  analysis  and  synthesis 
of  discrete-event  supervisory  control  algorithms. 
Specifically,  performance  indices  of  supervisors  can  be 
defined  in  terms  of  the  language  measure. 

The  signed  real  measure  for  a  DFSA,  presented  in  this 
paper,  is  constructed  based  on  assignment  of  an  event 
cost  matrix  and  a  characteristic  vector.  Two  techniques 
for  language  measure  computation  have  been  recently 
reported.  While  the  first  technique  (Wang  and  Ray 
2004)  leads  to  a  system  of  linear  equations  whose 
(closed  form)  solution  yields  the  language  measure 
vector,  the  second  technique  (Ray  and  Phoha  2003)  is 
a  recursive  procedure  with  finite  iterations.  A  sufficient 
condition  for  finiteness  of  the  signed  measure  has  been 
established  in  both  cases;  and  an  upper  bound  is 
established  for  the  max  norm  of  the  language  measure 
vector. 

In  order  to  induce  total  ordering  on  the  measure  of 
different  sublanguages  of  a  plant  language  under 
different  supervisors,  it  is  implicit  that  same  strings  in 
different  sublanguages  must  be  assigned  the  same 
measure.  This  is  accomplished  by  a  quantitative  tool 
that  requires  a  systematic  procedure  to  assign  a 
characteristic  vector  and  an  event  cost  matrix.  The 
clarifications  and  extensions  presented  in  this  paper 
are  intended  to  enhance  development  of  a  systematic 
analytical  tool  for  synthesizing  discrete-event 
supervisory  control.  For  example,  Ray  et  al.  (2004) 
have  proposed  unconstrained  optimal  control  of  regular 
languages  where  a  state-based  optimal  control  policy  is 
obtained  by  selectively  disabling  controllable  events  to 
maximize  the  measure  of  the  supervised  plant  language. 

The  paper  is  organized  in  eight  sections  including  the 
present  introductory  section  and  two  appendices. 
Section  2  briefly  describes  the  language  measure  and 
introduces  the  notations.  Section  3  presents  the  proce¬ 
dure  by  which  the  performance  of  different  supervisors 
can  be  compared  based  on  a  common  quantitative 
tool.  It  also  discusses  two  methods  for  computing 
language  measure.  Section  4  addresses  issues  regarding 
physical  interpretation  of  the  event  cost  used  in  the 
language  measure.  Section  5  presents  a  recursive 
algorithm  for  identification  of  the  language  parameters 
(i.e.,  elements  of  the  event  cost  matrix).  Section  6 
illustrates  the  usage  of  the  language  measure  for  con¬ 
struction  of  metric  spaces  of  formal  languages  and 
synthesis  of  optimal  discrete-event  supervisors.  Section  7 
presents  an  application  of  the  language  measure  on  the 
discrete-event  model  of  a  twin-engine  unmanned 


aircraft  (Ray  and  Phoha  2003,  Ray  et  al.  2004). 
The  paper  is  summarized  and  concluded  in  §8  along 
with  recommendations  for  future  research.  Appendix  I 
provides  pertinent  mathematical  background  of  mea¬ 
sure  theory  as  needed  in  the  main  body  of  the  paper. 
Appendix  II  establishes  a  sufficient  condition  for  abso¬ 
lute  convergence  of  the  language  measure. 


2.  Language  measure  concept 

This  section  first  introduces  the  signed  real  measure  of 
regular  languages,  originally  reported  in  (Ray  and 
Phoha  2003,  Wang  and  Ray  2004).  Then,  the  underlying 
concepts  of  language  measure  are  clarified  in  the  context 
of  discrete  event  supervisory  (DES)  control. 

Let  G,  =  (g,  E,  <5,  qt,  Qm)  be  a  trim  (i.e.,  accessible  and 
co-accessible)  finite-state  automaton  model  (Ramadge 
and  Wonham  1987,  Cassandras  and  Lafortune  1999) 
that  represents  the  discrete-event  dynamics  of  a  physical 
plant,  where  Q  —  {<y^ :  k  e  Xq }  is  the  set  of  states  and 
Iq  =  {1,2, ...,«}  is  the  index  set  of  states;  the  automa¬ 
ton  starts  with  the  initial  state  </,■;  the  alphabet  of 
events  is  E  =  {cta-:  k  e  IE),  and  =  {1,2, ...  ,£}  is  the 
index  set  of  events;  8:  Q  x  E  —>  Q  is  the  (possibly 
partial)  function  of  state  transitions;  and  Qm  = 
{qmi,  qmi,  ■  ■  ■ ,  <lmr}  ^  Q  is  the  set  of  marked  (i.e., 
accepted)  states  with  qmk  —  qj  for  some  j  e  Xq. 

Let  E*  be  the  Kleene  closure  of  E,  i.e.,  the  set  of  all 
finite-length  strings  made  of  the  events  belonging  to  E 
as  well  as  the  empty  string  e  that  is  viewed  as  the 
identity  of  the  monoid  E*  under  the  operation  of 
string  concatenation,  i.e.,  es  —  s  —  se.  The  extension 
8:  Q  x  E*  -*  Q  is  defined  recursively  in  the  usual  sense 
(Martin  1997,  Hopcroft  et  al.  2001).  For  DES  control 
(Ramadge  and  Wonham  1987),  the  event  alphabet  E  is 
partitioned  into  sets,  Ec  and  E  —  Ef  of  controllable 
and  uncontrollable  events,  respectively,  where  each 
event  in  Ec  and  no  event  in  E  —  Ec  can  be  disabled  by 
the  supervisor. 

Definition  1:  The  language  L(G;)  generated  by  a 
DFSA  Gj  initialized  at  the  state  qt  e  Q  is  defined  as 

L(Gd  =  [s  e  E*  I  8(qt,s)  e  Q).  (1) 

Since  the  state  transition  function  8  is  allowed  to  be 
a  partial  function,  L{G,)  c  E*  following  Definition  1; 
if  8  is  a  total  function,  then  the  generated  language 
L(Gi)  =  E*. 

Definition  2:  Given  a  DFSA  plant  model  G„  having 
the  set  of  controllable  events  Ec  c  E,  let  5  and  S  be  two 
controllable  supervisors  (i.e.,  each  of  S  and  S  is  repre¬ 
sented  by  an  event  disabling  mapping  L(G,)  — >•  2S<). 
Let  the  languages  of  the  plant  supervised  by  S'  and  S 
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be  denoted  as  L(S/Gj)  and  L{S/Gi),  respectively.  Then,  5 
is  said  to  be  less  permissive  (or  more  restrictive)  than  S, 
denoted  as  S  <  S,  if  the  following  condition  holds. 

S  <S  if  US/Gi)  c  LiS/G,).  (2) 

In  other  words,  S  may  disable  a  larger  set  of  control¬ 
lable  events  than  5  following  the  execution  of  an  event 
string  seE*. 

Definition  3:  The  language  Lm(G,)  marked  by  a  DFSA 
Gi,  initialized  at  the  state  qt  e  Q,  is  defined  as 

Lm{Gi)  =  {jeE*  |  S(qi,s)e  Qmj.  (3) 

Definition  4:  For  every  q,,  qk  e  Q,  let  L,y.  denote  the 
set  of  all  strings  that,  starting  from  the  state  qt,  termi¬ 
nate  at  the  state  qk,  i.e., 

L#  =  {se  E*  |  8(qt,s)  =  qk}.  (4) 

In  order  to  obtain  a  quantitative  measure  of  the  marked 
language,  the  set  Qm  of  marked  states  is  partitioned  into 
Qm  and  Q~,  i.e.,  Qm  =  Q+  U  Q~  and  Q+  n  Q~  =  0. 
The  positive  set  contains  all  good  marked  states 
that  one  would  desire  to  reach,  and  the  negative  set 
Q~  contains  all  bad  marked  states  that  one  would  not 
want  to  terminate  on,  although  it  may  not  always  be 
possible  to  completely  avoid  the  bad  states  while 
attempting  to  reach  the  good  states.  From  this  perspec¬ 
tive,  each  marked  state  is  characterized  by  an  assigned 
real  value  that  is  chosen  based  on  the  designer’s  percep¬ 
tion  of  the  state’s  impact  on  the  system  performance. 


bad  strings.  Partitioning  Qm  into  the  positive  set  g+ 
and  the  negative  set  Q~  leads  to  partitioning  of  the 
marked  language  Z.,„(G,)  into  a  positive  language 
L+(G,)  and  a  negative  language  L~(Gi).  Based  on  the 
equivalence  classes  defined  in  the  Myhill-Nerode 
Theorem  (Hopcroft  et  al.  2001),  the  regular  languages 
L(Gj)  and  L,„(G,)  can  be  expressed  as 

L(Gi)  =  U  Luk  (6) 

k^XQ 

Lm(G;)  =  L+(GY)  [J  Lm{Gt)  (7) 

where  the  sublanguage  c  L(G;)  is  uniquely  labelled 
by  the  state  q/{,kelQ  and  Liyk  n  Ljj  =  0  Vk  /  j; 
and  L+(Gi)  =  U qkeQ+Liyk  and  L~(Gi)  =  U qk€Q-LiJc  are 
good  and  bad  sublanguages  of  L„,(G;),  respectively. 
Then,  the  null  sublanguage  LQ(G,) —  Gqk^QmLj^  and 
L{Gi)  —  L°(Gj)  U  L+(G;)  U  L~(Gj). 

Now  a  signed  real  measure  is  constructed  as  /F: 
2l(g‘)  — r  =  (_oo,  oo )  on  the  a-algebra  M  =  2L^Gi\ 
(Appendix  I  provides  details  of  measure-theoretic 
definitions  and  results.)  With  this  choice  of  cr-algebra, 
every  singleton  set  made  of  an  event  string  s  e  T(G,) 
is  a  measurable  set,  which  allows  its  quantitative 
evaluation  based  on  the  above  state-based  decomposi¬ 
tion  of  L(G,)  into  null  (i.e.,  L0(G;)),  positive  (i.e., 
L+(G,)),  and  negative  (i.e.,  L~n(G,))  sublanguages. 

Conceptually  similar  to  the  conditional  probability, 
each  event  is  assigned  a  cost  based  on  the  state  at 
which  it  is  generated. 

Definition  6:  The  event  cost  of  the  DFSA  G,  is  defined 
as  a  (possibly  partial)  function  it:  E*x  Q  — >  [0, 1]  such 
that  Vr/,  e  Q ,  Ver,-  e  E,  Vs  e  E*, 


Definition  5:  The  characteristic  function  /:  Q  — »■ 
[— 1,  1]  assigns  a  signed  real  weight  to  a  state-based  sub¬ 
language  Ljj,  having  each  of  its  strings  terminating  on 
the  same  state  qj,  and  is  defined  as 


7t[(Tj,  r/,]  =  0  if  8(qj,<jj  )  is  undefined;  jf[e,  qj\  —  1; 


n[cTj,  qi\  =  TCjj  e  [0, 1);  ^  %  <  1; 

jelz 


(8) 


V?,-  e  Q ,  xiqj)  e 


[-1,0), 

9}  e  Qm 

{0}, 

Qj  £  Qm 

(5) 

(0, 1], 

Qj  e  Qm- 

vector. 

denoted  by 

X  = 

led  the 

X-vector, 

where 

Xj  =x{qj)-  That  is,  the  yth  element  xj  of  X-vector  is  the 
weight  assigned  to  the  corresponding  state  qj. 

In  general,  the  marked  language  T„,(G,)  consists  of 
both  good  and  bad  strings,  which  start  from  the  initial 
state  qt,  respectively  lead  to  2+  and  Qm.  Denoting  the 
set  difference  operation  by  any  event  string  belong¬ 
ing  to  the  language  T°(G,)  =  T(G,)  —  L,„(G,)  leads  to 
one  of  the  non-marked  states  belonging  to  Q  —  Qm 
and  L°(Gj )  does  not  contain  any  one  of  the  good  or 


n[cTjS,  qi\  —  n[cTj,  qt\  ft[s,  8 {qj,  cry)]. 


A  simple  application  of  the  induction  principle  to 
the  last  part  of  Definition  6  shows  n[st,  qj\  —  ?f[^,  q/\ 
n[t,8(qj,s)].  The  condition  J2keie  <  *  provides  a 
sufficient  condition  for  the  existence  of  the  real 
signed  measure  as  discussed  in  §3  and  Appendix  II. 
Additional  comments  on  the  physical  interpretation  of 
the  event  cost  are  provided  in  §  4. 

The  n  x  l  event  cost  matrix  is  defined  as 


7fn 

H 12  ■ 

■  K\l 

7T21 

JC22  ■ 

■  Kit 

n  = 

-  Hnl 

Hill 

Hnt  - 

(9) 
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Definition  7:  The  state  transition  cost,  tt.QxQ 
[0, 1),  of  the  DFSA  G,-  is  defined  as  follows. 

Vz', ./'  e  Tq, 

_  Eo-ej:  ?<•]’  if  CT)  =  9]  , 

n"  lO  if  {%„ct)  =  %}  =  0- 

The  n  x  n  state  transition  cost  matrix  is  defined  as 


n  li 

7Ti2 

•  H 1  n 

^21 

H22 

^2  n 

-  ttn\ 

tt/i2 

•  H/in 

(11) 


and  is  referred  to  as  the  II-matrix  in  the  sequel. 

Definition  8:  Given  a  DFSA  G,  =  (Q,  E,  <5,  qt,  Q„)  the 
cost  vl  of  a  sublanguage  K  c  L(G,)  is  defined  as  the 
sum  of  the  event  cost  if  of  individual  strings  belonging 
to  K. 


!/'(*)  =  £*[*,?,■].  (12) 

seK 

Definition  9:  For  a  given  DFSA  G,,  the  signed  real 
measure  of  every  singleton  string  set  j.v)  e  Ljj  c  L(G,) 
is  defined  as  m‘(M)  =  jt(j,  <y,)x/  implying  that 


e  LLj,  mXM) 


—  (f  9j  £  Qm 

>  0,  qj  e  0+ 

<  0,  qj  e 


(13) 


Thus  an  event  string  terminating  on  a  good  (bad) 
marked  state  has  a  positive  (negative)  measure  and 
one  terminating  on  a  non-marked  state  has  zero 
measure.  It  follows  from  Definition  9  that  the  signed 
measure  of  the  sublanguage  Ltj  c  L{G,)  of  all  events, 
starting  at  qt  and  terminating  at  qj,  is 


MU/j) 


\  xj 

Ve/-  ) 


(14) 


Wang  and  Ray  2004).  The  total  variation  measure  |/T| 
of  m'  has  also  been  shown  to  be  finite  for  every  i  e  Tq 
(Ray  and  Phoha  2003). 

In  the  above  setting,  the  role  of  the  language  measure 
in  DES  control  synthesis  is  explained  below. 

A  discrete-event  non-marking  supervisor  S  restricts  the 
marked  behaviour  of  an  unsupervised  ( i.e .,  uncontrolled) 
plant  Gi  such  that  Lm(S/Gj)  C  Lm(Gj).  The  unsupervised 
marked  language  Lm(G ,)  consists  of  good  strings  leading 
to  Qf  and  bad  strings  leading  to  Qm.  A  supervised  {i.e., 
controlled)  language  Lm(S/Gi)  based  on  a  given  specifica¬ 
tion  of  the  supervisor  S  may  disable  some  of  the  bad  strings 
and  keep  some  of  the  good  strings  enabled.  Different  super¬ 
visors  Sj\ y  e  {1,2, ...  ,ns]  for  a  DFSA  G,  achieve  this  goal 
in  different  ways  and  generate  a  partially  ordered  set  of 
supervised  sublanguages  { Lnl{Sj/Gj ):  j  e  { 1,2, .. .  ,ns}}.  The 
real  signed  measure  ff  provides  a  precise  quantitative  com¬ 
parison  of  the  controlled  plant  behaviour  under 
different  supervisors  because  the  set  {fi(Lm(Sj/Gi)): 
y  e  { 1,2,...,  «v}}  is  totally  ordered. 

In  order  to  realize  the  above  goal,  the  performance  of 
different  supervisors  has  to  be  evaluated  based  on  a 
common  quantitative  tool.  Let  G=  {QG,'£,8G,qf,Q^1) 
denote  the  unsupervised  plant  and  S={Qs,  E,<55, 
qf,  Qg)  denote  the  supervisor  with  respective  languages 
L(G )  and  L(S)  and  the  corresponding  marked  languages 
Lm(G)  and  Lm(S). 

Let  G  =  ( Q ,  E,  8,  q\ ,  Qm)  where  Q  —  QGx  Qs, 
01  =  («?,«?),  Qm  =  {(p,p)\p  e  Qg  and  p  e  Q?J  and 
the  transition  function  8  is  defined  by  the  formula: 
Wp  e  Qg,  p  e  Qs,  and  a  e  E 

8((p,p),a)=  ( 8G{p,a),8s{p,a )).  (16) 

Then,  the  marked  language  T,„(G)  of  the  automaton  G 
is  Lm(G)  fl  Lm(S )  because  G  is  a  parallel  composition 
(Ramadge  and  Wonham  1987,  Cassandras  and 
Lafortune  1999)  of  the  automata  G  and  S  that  have 
the  common  alphabet  E.  Then,  it  follows  that  the  exten¬ 
sion  8  satisfies  the  condition 


Definition  10:  The  signed  real  measure  of  the  language 
of  a  DFSA  Gi  initialized  at  a  state  qi  e  Q ,  is  defined  as 

/x,  =  fi(L{Gi))  =  ^  ffifff.  (15) 

j&Q 

The  language  measure  vector,  denoted  as  p  — 
[/xi  M2  ■  ■  ■  M;i]r>  is  called  the  /z-vector. 

Remark  1:  ff{Lm{Gi))  —  m,  Vz  e  Tq  because  Xk  =  0 
Vc/z  e  Q-  Qm. 

It  follows  from  Definition  10  that  ff(Lij)  =  vl(Ljj)xj. 
Under  the  condition  of  ffk7tjk  <  1  in  Definition  6, 
convergence  of  the  signed  real  language  measure 
m'  has  been  proved  in  (Ray  and  Phoha  2003, 


V.v  e  E*,  8{{p,p),  s)  =  ( SG{p ,  s),  8s (p,  s))  (17) 

whenever  8G(p,  s )  and  8s (p,  s )  are  defined. 

The  unsupervised  plant  language  L{G)  is  partitioned 
by  LG j,  \<j<nG  where  \QG\  —  nG .  Similarly,  the 
supervisor  language  L{S )  is  partitioned  by  Lfk, 
1  <  k  <  ns  where  |QS|  =  ns .  With  this  construction, 
each  of  the  sublanguages  LG .  is  further  partitioned  by 
LGjC\Lfk.  Thus,  for  any  qG  e  QG,  the  set  of  strings, 
which  is  retained  in  Lm{G )  fl  Lm(S),  is  given  by 
LG j  fl  (U qs€QSLfk).  In  this  setting,  the  goal  is  to  synthe¬ 
size  a  supervisor  that  will  retain  many  strings  that  termi¬ 
nate  on  some  state  in  QG+  while  discarding  many  strings 
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that  terminate  on  QG~  ■  It  will  yield  a  relatively  high 
measure  and  hence  good  performance. 

The  above  construction  shows  how  the  event  cost  and 
characteristic  function  assigned  to  the  unsupervised 
plant  can  be  used  as  a  quantitative  tool  with  which  the 
performance  of  different  supervisors  can  be  evaluated 
and  compared.  The  following  procedure  indicates  how 
this  can  be  accomplished  explicitly. 

Definition  11:  Let  G,  S  and  G  be  defined  as  above.  Let 
G  represent  the  unsupervised  plant  and  it0  be  the  event 
cost  function  and  xG  be  the  characteristic  function. 
Then,  for  the  DFSA  G  which  represents  the  language 
of  the  supervised  plant,  the  event  cost  function  it  is 
defined  as 


{o',  (??,?/)  =  nG[o,qG] 

Vo-  £  E  and  Vi,/  s.t.  1  <  i  <  nG,  1  <j<  ns 


The  x-vector  for  the  DFSA  G  is  defined  as 

x((«?.9?))=X°<«?W(9?)  (19) 


where  J{  ■ )  is  the  indicator  function  defined  as 


J(.P)  = 


1  P  e  Qsm 
0  PiQl 


(20) 


Let  s  e  L((9g  9s);  (?gi?S)>,  i.e.,  the  set  of  all  strings  starting 
at  the  state  (qG ,  qf)  £  Q  =  QG  x  Qs  and  terminating  at 
(qG,qf)-  If  qf  =  8G(q\,s),  it  follows  from  Definition  9 
that  /x({^})  =  i iG[s,qG]xG(qG)  for  the  unsupervised  (i.e., 
uncontrolled)  plant.  Following  equations  (18)  and  (19), 
the  measure  of  the  supervised  (i.e.,  controlled)  plant 
becomes 


Ai(M)  =  x[s,  (qG,qS\)] 

=  nG[s,qG]xG(qG)j{qSk).  (21) 

In  other  words,  if  no  event  in  the  string  ^  is  disabled 
by  the  supervisor,  then  /xi({s})  in  the  supervised  plant 
automaton  G  remains  the  same  as  in  the  unsupervised 
plant  automaton  G;  otherwise,  /xi({^})  =  0.  Thus, 
Definition  1 1  guarantees  that  the  same  strings  in 
different  supervised  sublanguages  of  the  unsupervised 
plant  language  L(Gi)  are  assigned  the  same  measure. 
Hence,  the  performance  of  different  supervisors  can  be 
compared  with  a  common  quantitative  tool. 

Finally  to  conclude  this  section,  it  should  be  noted 
that  while  the  domain  (i.e.,  2L(-G^)  of  the  language 
measure  /a'  is  partially  ordered,  its  range  which  is  a 
subset  of  R  becomes  totally  ordered.  The  set  L(G,) 
with  the  a-algebra,  2L^G‘\  forms  a  measurable  space. 
In  principle,  any  measure  /r  can  be  defined  on  this 


measurable  space  to  form  a  measure  space  (i.e.,  the 
triple  (L(Gj),  2U'G<\  /a1)).  The  choice  of  the  signed 
language  measure,  as  given  by  Definitions  9  and  10, 
has  been  motivated  by  the  fact  that  it  bears  a  physical 
significance  and  hence  is  qualified  to  serve  as  a 
performance  measure  for  DES  controller  synthesis. 
Moreover,  defining  the  measure  in  this  way  also  leads 
to  simple  computational  procedures  as  discussed  in  the 
next  section  and  further  elaborated  in  §4. 


3.  Language  measure  computation 

Various  methods  of  obtaining  regular  expressions  for 
DFSAs  are  reported  in  Martin  (1997)  and  Hopcroft 
et  al.  (2001).  While  computing  the  measure  of  a  given 
DFSA,  the  same  event  may  have  different  significance 
when  emanating  from  different  states.  This  requires 
assigning  (possibly)  different  costs  to  the  same  event 
defined  on  different  states.  Therefore,  it  is  necessary  to 
obtain  a  regular  expression  which  explicitly  yields  the 
state-based  event  sequences.  In  order  to  compute  the 
language  measure,  it  is  convenient  to  transform  the  pro¬ 
cedures  of  evaluating  regular  expression  from  symbolic 
equations  to  algebraic  ones.  The  following  two  methods 
(Ray  and  Phoha  2003,  Wang  and  Ray  2004)  are 
presented,  in  detail,  for  language  measure  computation. 

3.1.  Method  I:  closed  form  solution 

This  section  presents  a  closed-form  method  to  compute 
the  language  measure  via  inversion  of  a  square  operator. 

Definition  12:  Let  L,  =  L(Gi),  i  e  1q,  denote  the 
regular  expression  representing  the  language  of  a 
DFSA  Gj  —  ( Q ,  E,  8,  qt,  Qm),  where  qt  is  the  initial  state. 

Definition  13:  Let  erj  denote  the  set  of  event(s)  a  e  E 
that  is  defined  on  the  state  qj  and  leads  to  the  state 
qk  £  Q,  where  j,  k  £  XQ,  i.e.,  %/,cr)  =  qk,  Va  £  o’j  c  E. 

Then,  given  a  DFSA  G,  =  ( Q ,  E,  8,  qt,  Qm)  the  procedure 
to  obtain  the  system  equation  by  a  set  of  regular  expres¬ 
sions  Li  of  the  language  L(Gi),  i  £  Tq,  is  as  follows: 

V <7/  £  g,  U  =  £  Rq  +  c,  (22) 

jeZQ 

where  the  operator  ^  indicates  the  sum  of  regular 
expressions  (equivalently,  union  of  regular  languages); 
and  R,j  is  defined  as  follows. 

If  there  exists  a  £  E  such  that  8(qi,  a)  —  qj  £  Q  for 
/  £  {1, . . . ,«},  then  Rij  —  rr/ Lj,  otherwise,  Rtj  =  0. 

The  set  of  symbolic  equations  may  be  written  as 

/,-  (23) 
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The  above  system  of  symbolic  equations  can  be  solved 
using  a  result  given  below,  which  is  illustrated  through 
an  example. 

Lemma  1:  Let  u,  v  be  two  known  regular  expressions 
and  r  be  an  unknown  regular  expression  that  satisfies 
the  following  algebraic  identity : 

r  —  ur  +  v.  (24) 

Then,  the  following  relations  are  true. 

(1)  r  —  u*v  is  a  solution  to  equation  (24). 

(2)  If  e  <£.  u,  then  r  —  u*v  is  the  unique  solution  to 
equation  (24). 

Proof:  The  proof  of  Lemma  1 ,  which  is  also  known  as 
Arden’s  relation,  is  given  in  Yu  (1997)  and  Ray  and 
Phoha  (2003).  □ 

Example  1:  In  this  example,  shown  in  figure  1,  the 
alphabet  is  E  =  {a,  b};  the  set  of  states  is  Q  —  {1,2, 3}; 
the  initial  state  is  1;  and  the  only  marked  state  is  2. 
Let  the  set  of  linear  algebraic  equations  representing 
the  transitions  at  each  state  of  the  DFSA  be  as  follows: 


Proof:  Following  equation  (22)  and  Definition  5 

Vi  e  IQ,  i-i'(e)  =  xa  (27) 

Therefore,  each  element  of  the  vector  X  = 
[/I  Xi  ■  ■  ■  XnY  is  the  forcing  function  in  equations 
(23)  and  (24).  Starting  from  the  state  qh  the  measure 
of  the  language  L,  =  L(G,)  (see  Definition  12) 


=  X]  +  v‘(s)Xi 

j 


=  X]  7r(rT/)F/(L/‘)  +  Xi 
j 

=  X^OTA  +  X/- 


L\  —  a\L\  +  b\L2  +  e 
Li  —  a\L\  T  b^L^  -f-  e 
Lt,  —  a^L\  -f-  b^Li  4-  6 


(25) 


where  the  ‘forcing’  term  e  is  introduced  on  the  right 
side  of  each  equation.  For  example,  by  application  of 
Lemma  1,  the  regular  expression  for  the  language 
L(G\)  is  given  as 

L\  —  (a\)*  b\(a\{a\)*  b\  +  b\a\(a\)*b\  +  b\b\f  +  6. 

Instead  of  obtaining  regular  expressions,  the  language 
measure  can  be  directly  computed  by  transforming 
this  set  of  equations  into  a  system  of  linear  equations 
based  on  the  following  result. 

Theorem  1:  Following  Definition  10,  the  language 
measure  of  the  symbolic  equation  (23)  is  given  by 


The  third  equality  in  the  above  derivation  follows  from 
the  fact  that  e  fi  a- Lj  —  0.  It  is  also  true  that 

Vjfk,  crl Lj  of  Lk  —  0  (28) 

since  each  string  in  aj Lj  starts  with  an  event  in  of  while 
each  string  in  of  Lk  starts  from  an  event  in  of  and 
aj  fl  of  =  0  V/Y  k  because  G',  is  a  DFSA.  This  justifies 
the  fourth  equality.  The  fifth  equality  follows  from 
Definition  8  and  the  fact  that  /r'(L,j)  =  u'(L,j)x(^); 
therefore,  by  Definitions  7  and  13,  ii\a\Lf)  —  n[qi,qj\ 
pJ{Lj)  —  7i  j  /Xj .  LJ 

In  vector  notation,  equation  (26)  in  Theorem  1  is 
expressed  as 


p  —  II/i  +  X 


Pi  =  X]  TtijPj  +  Xi- 
j 


(26)  whose  solution  is  given  by 


a*  =  (i  —  n)_1x 


(29) 


Figure  1 .  Finite  state  machine  for  Example  1 . 


provided  that  the  matrix  I  —  II  is  invertible.  The  follow¬ 
ing  important  result  guarantees  the  existence  of  p. 

Theorem  2:  Given  DFSAs  Gi=  {Q,T*,8,qi,Qm),  with 
the  state  transition  cost  matrix  II,  the  matrix  (i-n)  is 
an  invertible  bounded  linear  operator  and  p  e  R". 

Proof:  It  follows  from  Definitions  6  and  7  that  the 
induced  max  norm  PTH^  =  max,-  V  7r,y  =1—0  where 
6  e  (0, 1).  Then  (I  —  IT)  is  invertible  and  is  a  bounded 
linear  operator  and  ||(I  —  IT)-1  <  0-1  (Naylor  and 
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Sell  1982).  Then,  it  follows  from  equation  (29)  that 
H  e  R".  □ 

Corollary  1  (to  Theorem  2):  The  language  measure 
vector  n  is  bounded  as  ||/f|loo  <  0_1  where  9  = 
(i  -  liniioo). 

Proof:  The  proof  follows  by  applying  the  norm 
inequality  property  and  Theorem  2  to  equation  (29) 
and  the  fact  that  the  max  norm  HXH^  <  1  by 
Definition  5.  □ 

Alternatively,  sufficient  conditions  for  convergence  of  /x 
can  be  obtained  based  on  the  properties  of  nonnegative 
matrices  that  are  given  in  Appendix  II.  Therefore, 
Definitions  6  and  7  provide  a  sufficient  condition  for 
the  language  measure  //  of  the  DFSA  G,  to  be  finite. 
A  closed-form  algorithm  to  compute  a  language  mea¬ 
sure  based  on  the  above  procedure  is  presented  below. 

Algorithm  1:  Closed-form  computation  of  the 
language  measure 

(1)  For  a  given  G,:  =  ( Q ,  E,  8 ,  qu  Qm ),  specify  the  charac¬ 
teristic  vector  X  (see  Definition  5)  and  determine  the 
event  cost  matrix  n  (see  Definition  6)  via  experimen¬ 
tation  or  simulation,  as  described  later  in  §  5). 

(2)  Generate  the  Il-matrix  (Definition  7). 

(3)  Compute  the  language  measure  vector  //  •<— 
(I  —  II)_1X  using  Gaussian  elimination. 

(4)  Obtain  /x,-,  the  zth  element  of  //-vector,  which  is 
the  measure  of  the  generated  language  of  the 
DFSA  Gj. 

The  /th  element  of  the  zth  row  of  the  (I  —  II)'1  matrix, 
denoted  as  vj,  is  the  language  measure  of  the  DFSA 
with  the  same  state  transition  function  8  as  Gj  and 
having  the  following  properties:  (i)  the  initial  state  is 
z/,-;  (ii)  cjj  is  the  only  marked  state;  and  (iii)  the  /-value 
of  qj  is  equal  to  1.  Thus,  /x,-  =  /x(L(G,))  is  given  by 
di  —  ^jvjxj-  Numerical  evaluation  of  the  language 
measure  of  the  automaton  Gj  requires  Gaussian  elimina¬ 
tion  of  the  single  variable  /x,  involving  the  real  invertible 
matrix  (I  -  n).  Therefore,  the  computational  com¬ 
plexity  of  the  language  measure  algorithm  is  polynomial 
in  the  number  of  states. 


3.2.  Method  II:  recursive  solution 

This  section  presents  a  second  method  to  compute  the 
language  measure  using  a  recursive  procedure  based 
on  Kleene’s  theorem  (Martin  1997)  which  states  that 
the  marked  (i.e.,  accepted)  language  of  a  DFSA  is 
regular.  It  also  yields  an  algorithm  to  recursively  con¬ 
struct  the  regular  expression  of  its  language  instead  of 
the  closed  form  solution  in  Method  I. 


Definition  14:  Given  q,,  qk  eg,  a  non-empty  string  p 
of  events  (i.e.,  p  ^  e)  starting  from  qt  and  terminating 
at  qk  is  called  a  path.  A  path  p  from  qt  to  qk  is  said  to 
pass  through  qj  if  there  exists  s  ^  e  and  t  /  e  such  that 
p  =  sf,  8{qt,  s)  =  qj  and  <%,  t)  =  qk. 

Definition  15:  A  path  language  p[k  is  defined  to  be  the 
set  of  all  paths  from  qt  to  qk,  which  do  not  pass  through 
any  state  qr  for  r  >  j,  and  p\k.  The  path  language pik  is 
defined  to  be  the  set  of  all  paths  from  qt  to  qk.  Thus,  the 
language  Lik  is  obtained  in  terms  of  the  path  language 
Pik  as 


Li*  = 
=>■  v(Liy k)  — 


Pn  UleK 
Pik, 

v(pa)  +  1, 
v{Pik), 


if  k  —  i 
if  k  i 
if  k  —  i 
if  k  /  i. 


Every  path  language  p\k  is  a  regular  language  and  is  a 
subset  of  L(Gj).  As  shown  in  Ray  and  Phoha  (2003), 
following  recursive  relation  holds  for  0<j<n-  1, 
where  ||Q||  =  n. 


Theorem  3:  Given  a  DFSA  G,  =  {Q,T,,8,qi,Qm),  the 
following  recursive  relation  holds  for  1  <j<n- 1 


P°k  =  e  E:  8{q,a)  =  qk} 

Pik'  =Pbc\Jp'u+i(pj+ij+i)  Pj+ijc- 


(30) 


Proof:  Since  the  states  are  numbered  form  1  to  n  in 
increasing  order,  p<jk  =  {a  e  E:  8{qi,a)  —  qk]  follows 
directly  form  the  state  transition  map  8:  Q  x  E  — ^  Q 
and  Definition  15. 

Given  p\k  c  pjk  1 ,  let  us  consider  the  set  pjkl  —  p\k 
in  which  each  string  passes  through  qj+ 1  in  the  path 
from  q/  to  qk  and  no  string  must  pass  through  qm  for 
m  >  (j  +  1).  Then,  it  follows  that 


Pik 1  -Pik=PJu+iPJjtlk 


where  p^\k  can  be  expanded  as 


pj:u  -  ( 


pj+u+ipjti* 


,)  [jp'j+ik 


that  has  a  unique  solution  following  Theorem  1  because 
e£pJj+ 1 /+1  based  on  Definition  15.  Therefore, 

Pik 1  =  Pik  U P’u+ 1  (pj+u+i)  Pj+uk ■  □ 


Based  on  the  three  lemmas  proved  below,  the  above 
relations  can  be  transformed  into  an  algebraic  equation 
conceptually  similar  to  Theorem  1  in  Method  I.  Along 
with  the  procedure  to  compute  the  language  measure 
it  is  established  that,  Vz  e  1q,  J2j=\  nii  <  1  is  a  sufficient 
condition  for  finiteness  of  /x. 
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Lemma  2:  v((/^)*(U#a^-))  e  [0, 1). 

Proof:  Following  Definitions  6  and  8,  v{p°kf)  e  [0, 1). 

Therefore,  by  convergence  of  geometric  series, 


The  second  step  in  the  above  derivation  follows  from 
fact  that  p\k  rP{j+\(P1j+\,j+\)*P1j+\,k  =  0-  The  third  steP 
follows  from  Definition  8  and  the  last  step  is  a 
consequence  of  Lemma  4.  □ 


because  v(P°kj)  =►  Ej&  “  v(P°kk)- 
Lemma  3:  v(pj+ ,  j+i)  e  [0, 1). 


e  [0, 1) 


□ 


Proof:  The  path  pj+]  /+1  may  contain  at  most  j  loops, 
one  around  each  of  the  states  q\,q2,  . . .  ,qj ■  If  the  path 
Pj+ij+i  does  not  contain  any  loop,  then  v(pj+1  .+1)  e 
[0, 1)  because  Vs  e  pj+]  +1,  u(.s)  <  1  and  each  of  5  origi¬ 
nates  at  state  j  +  1 . 

Next  let  us  suppose  that  there  is  a  loop  around  qt 
and  that  does  not  contain  any  other  loop;  this  loop 
must  be  followed  by  one  or  more  events  ok  generated 
at  qt  and  leading  to  some  other  states  qm  where  m  e 
{1, 1}  and  m  f  t.  By  Lemma  2,  v(pj+]  +1)  e 
[0, 1).  Proof  follows  by  starting  front  the  innermost 
loop  and  ending  with  all  loops  at  qj  □ 


Lemma  4: 


=  <31) 

Proof:  Since  v{pl+x  .+1)  e  [0, 1)  front  Lemma  3.3,  it 
follows  that 


<(^wr)  =  ,_  ‘  .[‘.oot 


□ 


Finally,  the  main  result  of  this  section  is  stated  as  the 
following  theorem. 

Theorem  4:  Given  a  DFSA  G,  =  (Q,  E,  <5,  q,,  Qm)  the 
following  recursive  result  holds  for  0  <j  <  n  —  1,  where 
HQII  =  n. 


=  v(Plk )  + 


y(plij+Mplj+vk) 

1  -  v(pj+  iJ+1) 


(32) 


Proof: 


v(pJikl )  -  v(pikUrij+i(Pmj+i)*Pj+i,k) 

-  v(Pik )  +  '•(/’/./.  i(/’/.:./.i)>/.i.a) 

=  v(pI)  +  v(pij+i)v{(pj+ij+iY)v(Pj+i*) 


=  v(Ptk )  + 


v(pJij+iHPj+ik) 

1  -  v(pJj+i,j+i) 


Based  on  the  above  result,  a  recursive  algorithm  to  com¬ 
pute  a  language  measure  is  presented  below. 


Algorithm  2:  Recursive  computation  of  the  language 
measure 


(1)  For  a  given  G,-  =  ( Q ,  £,  S,  qtl  Qm),  specify  the  charac¬ 
teristic  vector  X  (see  Definition  5)  and  determine  the 
event  cost  matrix  II  (see  Definition  6)  via  experimen¬ 
tation  or  simulation,  as  described  in  §  5). 

(2)  Compute  the  II-matrix  (Definition  7). 

(3)  v(p^k)< — 7T&  for  1  <l,k<n 

(4)  for  j  —  0  to  n  —  1 

for  l  —  \  to  n 
for  k  —  1  to  n 


v(Pikl )  =  v(Pik)+ 


v(p’,J+Mp'j+i,k ) 


1 


v(pl 


I'j+G+J 

end 

end 

end 

(5)  Calculate  v(Lj^)  front  v( pn()  using  Definition  15. 

(6)  n,< —  Eq  €Q„  v(Pij)Xi  is  a  measure  of  the  language 
U  of  the  DFSA  (?,. 


Since  there  are  only  three  for  loops,  the  computational 
complexity  of  the  above  algorithm  is  polynomial  in  the 
number  of  DFSA  states,  same  as  that  of  Algorithm  1 
in  Method  I. 


4.  Event  cost:  a  probabilistic  interpretation 

The  signed  real  measure  (see  Definition  10)  of  a  regular 
language  is  based  on  the  assignment  of  the  characteristic 
vector  X  (see  Definition  5)  and  the  event  cost  matrix  IT 
(Definition  6).  The  characteristic  vector  is  chosen  by  the 
designer  based  on  his/her  perception  of  the  individual 
state’s  impact  on  the  system  performance.  On  the 
other  hand,  the  event  cost  is  an  intrinsic  property  of 
the  plant.  The  event  cost  itjk  is  conceptually  similar  to 
the  state-based  conditional  probability  of  Markov 
Chains,  except  for  the  fact  that  it  is  not  allowed  to 
satisfy  the  equality  condition  ™jk  —  1  -  (Note  that 
Ek^jk  <  1  is  a  requirement  for  convergence  of  the 
language  measure.)  The  rationale  for  this  strict 
inequality  is  explained  below. 

Since  the  plant  model  is  an  inexact  representation  of 
the  physical  plant,  there  exist  unmodelled  dynamics 
to  account  for.  This  can  manifest  itself  either  as 
unmodelled  events  that  may  occur  at  each  state  or  as 
unaccounted  states  in  the  model.  Let  E"  denote  the  set 
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of  all  unmodelled  events  at  state  cjj  of  the  DFSA 
Gj  =  (Q,  E,  S,  cjj,  QJ.  Creating  a  new  unmarked  absorb¬ 
ing  state  qn+\,  called  the  dump  state  (Ramadge  and 
Wonham  1987),  and  extending  the  transition  function 
S  to  <5ext:  ( Q  U  {q„+ 1})  x  (E  U,  E")  ->(gu  {tf„+i}),  it 
follows  that 


Axth//,  rr) 


S(qj,  a),  if  qj  e  Q  and  a  e  E 
qn+ 1,  if  qj  e  Q  and  u  6  E" 

(/„+i,  if  j  —  n+\  and  a  e  E  (J  E". 


then  directly  related  (but  not  necessarily  equal)  to  the 
expected  value  of  the  characteristic  function. 

The  choice  of  the  characteristic  function  (see 
Definition  5)  is  based  on  the  importance  assigned  to 
the  individual  marked  states  of  the  DFSA.  Therefore, 
in  the  setting  of  the  language  measure,  a  supervisor’s 
performance  is  superior  if  the  supervised  plant  is  more 
likely  to  terminate  at  a  good  marked  state  and/or  less 
likely  to  terminate  at  a  bad  marked  state. 


(33) 


Therefore  the  residue  9j  —  1  —  T )k  denotes  the 
probability  of  the  set  of  unmodelled  events  Ej' 
conditioned  on  the  state  j.  The  II  matrix  can  be  similarly 
augmented  to  obtain  a  stochastic  matrix  naug  as  follows: 


TX\\ 

JTi2  . 

•  tt \n 

en 

7t2l 

7T22  ■ 

•  ^2  n 

e2 

(34) 

H n\ 

til 

•  Ttnn 

en 

_  0 

0  . 

.  0 

l . 

Since  the  dump  state  q,t+l  is  not  marked,  its  character¬ 
istic  value  Xji+i  =  x(/?n+i)  =  0.  The  characteristic  vector 
then  augments  to 

Xaug  =  [Xr  Of 

and,  with  these  extensions,  the  language  measure  vector 
J*aug  =  [Mi  4  2  ■■■  4«  Rn+  i]7=[f  M„+1]r  of  the  aug¬ 
mented  DFSA  Gaug  =  {Q  U  {qn+x},  E  U,  E“,  Sext,  qh  QJ 
can  be  expressed  as 


5.  Estimation  of  language  measure  parameters 

This  section  presents  a  recursive  algorithm  for  identifica¬ 
tion  of  the  language  measure  parameters  (Wang  et  al. 
2005)  (i.e.,  elements  of  the  event  cost  matrix  II)  (see 
Definition  6)  which,  in  turn,  allows  computation  of  the 
state  transition  cost  matrix  II  (see  Definition  7)  and 
the  language  measure  //-vector  (see  Definition  10).  It  is 
assumed  that  the  underlying  physical  process  evolves 
at  two  different  time  scales.  In  the  fast-time  scale,  i.e., 
over  a  short  time  period,  the  system  is  assumed  to  be 
an  ergodic,  discrete  Markov  process.  In  the  slowly- 
varying  time  scale,  i.e.,  over  a  long  period,  the  system 
(possibly)  behaves  as  a  non-stationary  stochastic  pro¬ 
cess.  For  such  a  slowly-varying  non-stationary  process, 
it  might  be  necessary  to  redesign  the  supervisory  control 
policy  in  real  time.  In  that  case,  the  Id-matrix  parameters 
should  be  updated  at  selected  slow-time  epochs. 

5.1.  A  recursive  parameter  estimation  scheme 


=  /  ft  \  _  /n/i  +  /x„+i[@i 

^  “  Wlj  V  4n+ 1 


Let  py  be  the  transition  probability  of  the  event  ay  at  the 


state  qt. 

oj' 

(35) 

Pa  = 

i.e., 

P[crj\qi\, 

0, 


if  3 q  e  Q,  s.t.  q  =  S(qh  a,) 
otherwise 


(36) 


Since  x(qn+\)  —  0  and  all  transitions  from  the  absorbing 
state  qn+\  lead  to  itself,  i.e.,  //,„+ 1  =  JLJG n+\))  =  0, 
equation  (35)  reduces  to  that  for  the  original  plant  Gj. 
Thus,  the  event  cost  can  now  be  interpreted  as 
conditional  probability,  where  the  residue  0j  — 
1  —  Y^,k  >  0  accounts  for  the  probability  of  all 
unmodelled  events  emanating  from  the  state  qj.  With 
this  interpretation  of  event  cost,  jr[.v,  qi]  (see  Definition 
6)  denotes  the  probability  of  occurrence  of  the  event 
string  s  in  the  plant  model  G ,  starting  at  state  qt  and 
terminating  at  state  8{s,q,).  Hence,  v'(Ljj )  (see 
Definition  8),  which  is  a  non-negative  real  number,  is 
directly  related  to  the  sum  of  probabilities  that  state  qt 
would  be  reached  via  alternative  paths  as  the  plant 
operates.  (Note  that  >  1  is  possible  if  Ly 

contains  multiple  strings.)  The  language  measure 
=  Ri{L{Gl))  -  T,jeia  vKLij)  =  L/eie  v\Li,j)Xj  is 


and  its  estimate  be  denoted  by  the  parameter  py  that  is 
to  be  identified  from  the  ensemble  of  simulation  and / 
or  experimental  data. 

Let  a  strictly  increasing  sequence  of  time  epochs  of 
consecutive  event  occurrence  be  denoted  as 

T  =  {4:  k  e  No],  (37) 

where  No  is  the  set  of  non-negative  integers.  Let  the  indi¬ 
cator  J.  No  xlg  xIE->  {0, 1}  represent  the  incident  of 
occurrence  of  an  event.  For  example,  if  the  DFSA  was 
in  state  q,  at  time  epoch  4_i,  then 


i kjih)  = 


if  cjj  occurs  at  the  time  epoch  4  e  T 
otherwise. 

(38) 
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Consequently,  the  number  of  occurrences  of  any  event  in 
the  alphabet  £  is  represented  by  'b:N0  x  Xq  — >  {0, 1}. 
For  example,  if  the  DFSA  was  in  state  qt  at  the  time 
epoch  tk~  1,  then 

T i(k)  =  J2  MQ-  (39) 

Let  n:  No  xlg  xIE->  No  represent  the  cumulative 
number  of  occurrences  of  an  event  at  a  state  up  to  a 
given  time  epoch.  That  is,  ny(k)  denotes  the  number  of 
occurrences  of  the  event  ay  at  the  state  q,  up  to  the 
time  epoch  tk  e  T.  Similarly,  let  N:  No  x  Xq  — >  No 
represent  the  cumulative  number  of  occurrences  of  any 
event  in  the  alphabet  £  at  a  state  up  to  a  given  time 
epoch.  Consequently, 

Ni(k)  =  nj{k).  (40) 

Jcx  y 

A  frequency  estimator,  py(k),  for  probability  py(k)  of  the 
event  ay  occurring  at  the  state  cp  at  the  time  epoch  tk,  is 
obtained  as 


lim  py(k)  =  py. 


(41) 


Convergence  of  the  above  limit  is  justified  because  the 
occurrence  of  an  event  at  a  given  state  of  a  stationary 
Markov  chain  can  be  treated  as  an  independent  and 
identically  distributed  random  variable. 

A  recursive  algorithm  of  learning  py  is  formulated  as  a 
stochastic  approximation  scheme,  starting  at  the  time 
epoch  to  with  the  initial  conditions:  py{ 0)  =  0  and 
/Zy(0)  —  0  for  all  i  6  IgJ  e  Is;  and  *F,(0)  =  0  for  all 
i  e  Xq.  Starting  at  k  —  0,  the  recursive  algorithm  runs 
for  {tk-  k  >  1}.  For  example,  upon  occurrence  of  an 
event  a,-  at  a  state  qh  the  algorithm  is  recursively 
incremented  as 


ny(k)  —  ny(k  -  1)  +  fy{k)  I 

N,(k)  =  Nfk-  1)  +  ¥f(fc).  J 


Next  it  is  demonstrated  how  the  estimates  of  the  lan¬ 
guage  parameters  (i.e.,  the  elements  of  event  cost 
matrix  O )  are  determined  from  the  probability  esti¬ 
mates.  As  stated  earlier  in  §4  the  set  of  unmodelled 
events  at  state  <y,,  denoted  by  £"  Vz  e  Xq,  accounts  for 
the  row-sum  inequality:  fZjTty  <  (see  Definition  6). 
Then,  P[£"]  =  9j  e  (0, 1]  and  J2,  Ty  =  1  -  ft-  An  esti¬ 
mate  of  the  (z'j')th  element  of  the  event  cost  matrix 
n  -matrix,  denoted  by  Tty,  is  approximated  as 


(43) 


Additional  experiments  on  a  more  detailed  automaton 
model  would  be  necessary  to  identify  the  parameters 
OjVielQ.  If  9i  <£  1,  the  problem  of  conducting 
additional  experimentation  can  be  circumvented  by  the 
following  approximation. 

A  single  parameter  0  ~  0,-  Vz  e  Xq,  i  e  Xq,  such  that 
0  <  0  <$C  1 ,  could  be  selected  for  convenience  of  imple¬ 
mentation.  From  the  numerical  perspective,  this  option 
is  meaningful  because  it  sets  an  upper  bound  on  the 
language  measure  based  on  the  fact  that  the  max 
norm  H/uH^  <  0_1.  Note  that  each  row  sum  in  the 
n  -matrix  being  strictly  less  than  1,  i.e.,  fZjTty  <  1,  is 
a  sufficient  condition  for  finiteness  of  the  language 
measure  (see  Appendix  II). 

Theoretically,  Tty  is  the  asymptotic  value  of  the 
estimated  probabilities  Tty(k)  as  if  the  event  a;-  occurs 
infinitely  many  times  at  the  state  qt.  However,  while 
dealing  with  finite  amount  of  data,  the  objective  is  to 
obtain  a  good  estimate  py  of  py  from  independent 
Bernoulli  trials  of  generating  events.  Critical  issues  in 
this  situation  are:  (i)  how  much  data  are  needed;  and 
(ii)  when  to  stop  if  adequate  data  are  available.  The 
next  section  5-B  addresses  these  issues. 


5.2.  Stopping  rules  for  recursive  learning 

A  stopping  rule  is  necessary  to  find  a  lower  bound  on  the 
number  of  experiments  to  be  conducted  for  identifica¬ 
tion  of  the  II-matrix  parameters.  This  section  presents 
two  stopping  rules  that  are  discussed  below. 

The  first  stopping  rule  is  based  on  an  inference 
approximation  having  a  specified  absolute  error  bound 
e  with  a  probability  k.  The  objective  is  to  achieve  a 
trade-off  between  the  number  of  experimental  observa¬ 
tions  and  the  estimation  accuracy. 

A  bound  on  the  required  number  of  samples  is 
estimated  using  the  Gaussian  structure  of  the  binomial 
distribution  that  is  an  approximation  of  the  sum  of  a 
large  number  of  independent  and  identically  distributed 
(i.i.d.)  Bernoulli  trials  of  if ,-;(?).  The  central  limit  theorem 
yields  Tty  ~  Jf(fty,  ify(l  —  Jty)/N),  where  Jf  indicates 
normal  (or  Gaussian)  distribution  with  E[fiy]  ~  Tty  and 
Var [if (/]  =  a2  ss  ify(l  —  ny)/N,  provided  thaj:  the  number 
of  samples  N  is  sufficiently  large.  Let  A  =  Tty  —  Tty,  then 
A/a  ~  A((0, 1).  Given  0  <  e  1  and  0  <  A«  1,  the 
problem  is  to  find  a  bound  Ny  on  the  number  N  of 
experiments  such  that  P{|A]  >  e}  <  k.  Equivalently, 


P 


o  ~  a 


<  k 


(44) 


that  yields  a  bound  Nb  on  N  as 


Nh> 


r'w 


2 

ify(l  ify)? 


Tty(k)  =  py(k)(\  -  9j)  V/'els. 


e 


(45) 
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where  f(x)  =  1  —  /2)  dt.  Since  the  parameter 

Tty  is  unknown,  one  may  use  the  fact  that  ;r,y(l  —  if y)  < 
0.25  for  every  ify  e  [0, 1]  to  (conservatively)  obtain  a 
bound  on  N  only  in  terms  of  the  specified  parameters 
s  and  A.  as 


Nh> 


(46) 


The  above  estimate  of  the  bound  on  the  required 
number  of  samples  is  less  conservative  than  that 
obtained  from  the  Chernoff  bound  and  is  significantly 
less  conservative  than  that  obtained  from  Chebyshev 
bound  which  does  not  require  the  assumption  of  any 
specific  distribution  of  A  except  for  finiteness  of  the 
rth  (r  —  2)  moment. 

The  second  stopping  rule,  which  is  an  alternative  to 
the  first  stopping  rule,  is  based  on  the  properties  of 
irreducible  stochastic  matrices.  Following  equation  (41) 
and  the  state  transition  function  <5  of  the  DFSA,  the 
state  transition  matrix  is  constructed  at  the  /rth  iteration 
as  P (k)  that  is  an  n  x  n  irreducible  stochastic  matrix 
under  stationary  conditions.  Similarly,  the  state  prob¬ 
ability  vector  p(A)  =  [pi(k)  pi (k)  ■  ■  ■  p„(k)]  is  obtained 
by  following  equation  (41) 


Pi(k) 


N,-(k) 

V,/,.  V/(A)' 


(47) 


Since  P(A)  is  a  stochastic  matrix,  i.e.,  YAj=\  kij(k)  —  1, 
and  P(/r)  is  irreducible,  there  is  a  unique  eigenvalue 
r  —  1  and  the  corresponding  left  eigenvector  p(/r)  (nor¬ 
malized  to  unity  in  the  sense  of  absolute  sum)  represent¬ 
ing  the  state  probability  vector,  provided  that  the  matrix 
parameters  have  converged  after  sufficiently  large 
number  of  iterations.  That  is, 

||p(fc)(I-P(A:))||00  <}-►  0  as/c^oo. 
k 


Equivalently, 


||(p(Ar)  —  p(/c  +  l))||oo  0  asAr-^oo.  (48) 

Taking  the  expected  value  of  ||p(A')  H,^  to  be  1  /n,  a 
threshold  of  q/n  is  specified,  where  n  is  the  number  of 
states  and  0  <  rj  <£  1  is  a  constant.  A  lower  bound  on 
the  required  number  of  samples  is  determined  from 
equation  (48)  as 


/Vstop  =  Integer  DA  (49) 

based  on  the  number  of  states,  n ,  and  the  specified 
tolerance  rj. 

6.  Usage  of  the  language  measure 


The  stopping  rule  makes  use  of  the  Perron-Frobenius 
Theorem  to  establish  a  relation  between  the  vector  p(k) 
and  the  irreducible  stochastic  matrix  P(A). 

Theorem  5:  Perron-Frobenius  Theorem  ( Senata  1973, 
Plemmons  and  Berman  1979 )  Let  P(A)  be  an  n  x  n 
irreducible  matrix,  then  there  exits  an  eigenvalue  r  such 
that 

(1)  r  e  R  and  r  >  0. 

(2)  r  can  be  associated  strictly  positive  left  and  right 
eigenvectors. 

(3)  r  >  A  V  eigenvalue  A  f  r. 

(4)  The  eigenvectors  associated  with  r  are  unique  to 
constant  multiples. 

(5)  If  0  <  B  <  P(k)  and  ji  is  an  eigenvalue  of  B ,  then 
m  <  |r|.  Moreover,  |/)|  =  r  implies  B  —  P(k). 

(6)  r  is  a  simple  root  of  the  characteristic  equation  of 
P(k). 

Corollary  1:  Corollary  to  Perron-Frobenius  Theorem 

n  n 

min  '^2  P[fk)  <  r  <  max  E  PiAk) 

'  1= i  '  l=i 

with  equality  on  either  side  implying  equality  throughout. 


The  two  methods  of  language  measure  computation, 
presented  in  §3,  have  the  same  computational  com¬ 
plexity,  0(>f),  where  n  is  the  number  of  states  of  the 
DFSA.  However,  each  of  these  two  methods  offer 
distinct  relative  advantages  in  specific  contexts.  For 
example,  while  the  closed  form  solution  in  §3.1  is 
more  amenable  for  analysis  and  synthesis  of  decision 
and  control  algorithms,  the  recursive  solution  in  §3.2 
might  prove  very  useful  for  construction  of  executable 
codes  in  real  time  applications.  The  following  two  sub¬ 
sections  present  usage  of  the  language  measure  for  con¬ 
struction  of  metric  spaces  of  formal  languages  and 
synthesis  of  optimal  discrete-event  supervisors. 

6.1.  Vector  space  of  formal  languages 

The  language  measure  can  be  used  to  construct  a 
vector  space  of  sublanguages  for  a  given  DFSA  G,-  = 
(Q,Y,,8,qi,Qm).  The  total  variation  measure  |/x| 
(Rudin  1987)  (see  Appendix  I)  induces  a  metric  on  this 
space,  which  quantifies  the  distance  function  between 
any  two  sublanguages  of  L(Gi). 

Proposition  1:  Let  L{G ,)  be  the  language  of  a  DFSA 
Gj  —  (Q,T,,8,qj,Qm).  Let  the  binary  operation  of 
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exclusive-OR  ©:  2i(G,)  x  2L(G,)  — >  2i(G,)  be  defined  as 

(A,  ®  K2)  =  (a,  [J  A2)  -  (^i  f]  A2)  (50) 

VK\,K2  c  L(Gj).  Then  (2L(G‘\@)  is  a  vector  space  over 
Galois  field  GF(2). 

Proof:  It  follows  from  the  properties  of  exclusive-OR 
that  the  algebra  (2i(Gi\©)  is  an  Abelian  group  where  0 
is  the  zero  element  of  the  group  and  the  unique  inverse 
of  every  element  K  c  2r(C,,}  is  K  itself  because  K\  © 
K2  —  0  if  and  only  if  K\  —  K2.  The  associative  and 
distributive  properties  of  the  vector  space  follows  by 
defining  the  scalar  multiplication  of  vectors  as: 
0  ©  K  =  0  and  1  <g>  K  =  K.  □ 

The  collection  of  singleton  languages  made  from  each 
element  of  L(G,)  forms  a  basis  set  of  vector  space 
(2l<g‘),©)  over  GF( 2).  It  is  shown  below,  how  total  var¬ 
iation  (Rudin  1987)  of  the  signed  measure  p  can  be  used 
to  define  a  metric  on  above  vector  space. 

Proposition  2:  Total  variation  measure  |/z|  on  2i(G4  is 
given  by  \/f(L)  =  ffsei.  HM)I  VA  c  L(Gt). 

Proof:  The  proof  follows  from  the  fact  that  £/d/x(A/c)| 
attains  its  supremum  for  the  finest  partition  of  L  which 
consists  of  the  individual  strings  in  L  as  elements  of  the 
partition.  □ 

Corollary  2  (to  Proposition  2):  Let  L(G ,)  be  the  lan¬ 
guage  of  a  DFSA  Gi  =  (Q,  £,  S,  qt,  Qm).  For  any 
K  e  2L(G>\  \p,\(K)  <  9~l  where  9=  1  -  Hn^  and  n  is 
the  state  transition  cost  matrix  of  the  DFSA. 

Proof:  The  proof  follows  from  Proposition  2  and 
Corollary  1.  □ 

Definition  16:  Let  A(G,)  be  the  language  of  a  DFSA 
Gj  =  {Q,T,,S,qi,Qm).  The  distance  function  d :  2L(G,)x 
2AG,) 

— >  [0,  oo)  is  defined  in  terms  of  the  total  variation 
measure  as  VA) ,  K2  c  UG,) 

d(K\,K2)  =  \p\((k1  [J  K2)  -  (a,  f]  A2))  (51) 

The  above  distance  function  d{  ■,  ■ )  quantifies  the  dif¬ 
ference  between  two  supervisors  relative  to  the  super¬ 
vised  performance  of  the  DFSA  plant. 

Proposition  3:  The  distance  function  defined  above  is  a 
pseudo-metric  on  the  space  2L{G‘> 

Proof:  Since  the  total  variation  of  a  signed  real 
measure  is  bounded  (Rudin  1987),  VA] ,  K2  c  A(G,), 
d(Ki,K2)—  |/x|(Ai  ©  A2)  e  [0, oo);  also  by  Definition 
16,  d(Ai, K2)  —  d(K2, Ai).  The  remaining  property  of 
the  triangular  inequality  follows  from  the  inequality 
| n | ( Ai  ©  K2)  <  |/z|(Ai)+  |/u|(A2)  which  is  based  on  the 


fact  that  (A,  ©  A2)  c  (A,  U  KT)  and  |/x|(A,)  <  |/x|(A2) 
VA  c  A2.  □ 

The  pseudo-metric  \fi\:  2L(Gi')  — >  [0,  oo)  can  be  converted 
to  a  metric  of  the  space  (2L(G‘\  ©)  by  clustering  all  lan¬ 
guages  that  have  zero  total  variation  measure  as  the 
null  equivalence  class  N  =  {A  e  2i(G,):  |/x|(A)  =  0}. 
This  procedure  is  conceptually  similar  to  what  is  done 
for  defining  norms  in  the  Lp  spaces.  In  that  case,  A f 
contains  all  sublanguages  of  A(G,),  which  terminate  on 
non-marked  states  starting  from  the  initial  state,  i.e., 
U  —  {0  u  (UqjgQmLij).  In  the  sequel,  \n\(-)  is  referred 
to  as  a  metric  of  the  space  2L{Gi) .  Thus,  the  metric 
|/x|(- )  can  be  generated  from  d(-,  •)  as: 

|  //.  |  (A)  =  d(K,J)  VA  e  2i(G,)  V/  e  A f.  Unlike  the  norms 
on  vector  spaces  defined  over  infinite  fields,  the  metric 
|/r|(-)  for  the  vector  space  (2L(Gi\  ©)  over  GF{ 2)  is  not 
a  functional.  This  interpretation  of  language  as  a 
vector  and  associating  a  metric  to  quantify  distance 
between  languages,  may  be  useful  for  analysis  and 
synthesis  of  discrete-event  supervisory  (DES)  control 
systems  under  different  settings. 

6.2.  Optimal  control  of  regular  languages 

The  (signed)  language  measure  //  could  serve  as  the  per¬ 
formance  index  for  synthesis  of  an  optimal  control 
policy  that  maximizes  the  performance  of  a  supervised 
sublanguage.  The  salient  concept  is  briefly  presented 
below. 

Let  S  =  {A0,  A1, . . . ,  SN}  be  a  set  of  supervisory  con¬ 
trol  policies  for  the  unsupervised  plant  automaton  G 
where  S°  is  the  null  controller  (i.e.,  no  event  is  disabled) 
implying  that  L(S°/G )  =  L{G).  Therefore,  the  controller 
cost  matrix  n(A°)  =  II0  that  is  the  Il-matrix  of  the 
unsupervised  plant  automaton  G.  For  a  supervisor 
Sk,  k  e  {1,2, . . . ,  N],  the  control  policy  is  required  to 
selectively  disable  certain  controllable  events  so  that 
the  following  (elementwise)  inequality  holds. 

n*  =  n (Sk)  <  n°  and  L(Sk/G)  c  L(G),  WSk  e  5. 

The  task  is  to  synthesize  an  optimal  cost  matrix 
IT  <  n°  that  maximizes  the  performance  vector 

n*  =  [i  —  irr'x,  i.e.,  p*  >  pk  =  [i  —  n'T'x  v  n/c  <  n° 

where  the  inequalities  are  implied  elementwise.  While 
the  details  of  the  underlying  theory  are  available  in 
recent  literature  (Ray  et  al.  2004),  a  synthesis  procedure 
for  optimal  control  of  regular  languages  is  succinctly 
presented  below. 

Let  the  DFSA  model  G  of  the  unsupervised  plant 
have  the  state  transition  cost  matrix:  II0  =  II  (see 
Definition  7)  and  the  characteristic  vector  X  (see 
Definition  5).  Then,  the  performance  vector  at  the 
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iteration  k  —  0  is  given  as 

A  =  bA  A  tf  =  (/-n°r'x 

where  the  /th  element  of  the  vector  /r°  is  the  perform¬ 
ance  of  the  unsupervised  plant  language,  with  state  qj  as 
the  initial  state.  Then,  /j.j  <  0  implies  that,  if  the  state  qj 
is  reached,  then  the  plant  will  yield  bad  performance 
thereafter.  Intuitively,  the  control  system  should  attempt 
to  prevent  the  automaton  from  reaching  qj  by  disabling 
all  controllable  events  that  lead  to  this  state.  Therefore, 
the  optimal  control  algorithm  starts  with  disabling  all 
controllable  events  that  lead  to  every  state  qj  for  which 
fXj  <  0.  This  is  equivalent  to  reducing  all  elements 
of  the  corresponding  columns  of  the  n°-matrix  by  dis¬ 
abling  those  controllable  events.  In  the  next  iteration, 
i.e.,  k—  1,  the  updated  cost  matrix  II1  is  obtained  as: 
IT1  =  IT0  -  A0  where  A°  >  0  (the  inequality  being 
implied  elementwise)  is  composed  of  event  costs  corre¬ 
sponding  to  all  controllable  events  that  have  been 
disabled. 

It  has  been  shown  in  Ray  et  al.  (2004)  that  /-1  < 
Hk  =  [I  —  1 IA]  1  X  elementwise  for  all  k  >  1 .  Although 
all  controllable  events  leading  to  every  state  correspond¬ 
ing  to  a  negative  element  of  /d  are  disabled,  some  of  the 
controllable  events  that  were  disabled  at  k  —  0  may  now 
lead  to  states  corresponding  to  positive  elements  of  /i1. 
Performance  could  be  further  enhanced  by  re-enabling 
these  controllable  events.  For  k  >  1,  II/c+l  ^Il^-l-A7' 
where  Aa  >  0  is  composed  of  the  state  transition  costs 
of  all  re-enabled  controllable  events  at  k.  It  is  also 
shown  in  Ray  et  al.  (2004)  that  the  number  of  iterations 
to  reach  optimality  does  not  exceed  the  number,  n,  of 
DFSA  states.  Therefore,  the  computational  complexity 
of  the  optimal  control  algorithm  is  polynomial  n. 

In  the  optimal  control  algorithm  in  Ray  et  al.  (2004), 
if  p°  >  0,  i.e.,  there  is  no  state  qj  such  that  0,  then 
the  plant  performance  cannot  be  improved  by  event  dis¬ 
abling  and  the  null  controller  5°  (i.e.,  no  disabled  event) 
is  the  optimal  controller  for  the  given  plant.  Therefore, 
the  cases  are  considered  where  /x°  for  some  state  qj. 

Starting  with  k  —  0  and  IT0  =  nplant,  the  control 
policy  is  constructed  by  the  following  two-step 
procedure: 

Step  1:  For  every  state  qj  for  which  /i.®  <  0,  disable 
controllable  events  leading  to  qj.  Now,  II1  =  IT0  —  A0, 
where  A0  >  0  is  composed  of  event  costs  corresponding 
to  all  controllable  events,  leading  to  qj  for  which  p*}, 
which  have  been  disabled  at  k  —  0. 

Step  2:  For  k  >  1,  if  /rA  >  0,  re-enable  all  controllable 
events  leading  to  qj,  which  were  disabled  in  Step  1.  The 
cost  matrix  is  updated  as:  IIA+I  =  IIA' +  Aa  for  k  >  1, 
where  A/c  >  0  is  composed  of  event  costs  corresponding 


to  all  currently  re-enabled  controllable  events.  The  itera¬ 
tion  is  terminated  if  no  controllable  event  leading  to  qj 
remains  disabled  for  which  /rA  >  0.  At  this  stage,  the 
optimal  performance  p*  =  [/  —  II*]_1X. 


7.  An  application  example 

Ray  and  Phoha  (2003)  and  Ray  et  al.  (2004)  have 
adopted  the  closed  form  method  of  language  measure 
(see  §3.1)  as  a  performance  index  for  optimal  super¬ 
visory  control  of  a  twin-engine  unmanned  aircraft  that 
is  used  for  surveillance  and  data  collection.  The 
language  measure  was  computed  and  verified  based  on 
both  closed  form  and  recursive  techniques  given  in  §3; 
the  results  were  identical  as  expected.  Engine  health 
and  operating  conditions,  which  are  monitored  in  real 
time  based  on  observed  data,  are  classified  into  three 
mutually  exclusive  and  exhaustive  categories: 

•  good; 

•  unhealthy  (but  operable); 

•  inoperable. 

In  the  event  of  any  observed  abnormality,  the  supervisor 
may  decide  to  continue  or  abort  the  mission.  The  finite 
state  automaton  model  of  the  plant  in  figure  2  has  13 
states  (excluding  the  dump  state),  of  which  three  are 
marked  states,  and  nine  events,  of  which  four  are 
controllable  and  the  remaining  five  are  uncontrollable. 
All  events  are  assumed  to  be  observable.  The  states 
and  events  of  the  plant  model  are  listed  in  table  1  and 
table  2,  respectively.  The  state  transition  function  S 
and  the  state-based  event  cost  Hy  (see  Definition  6)  are 
entered  simultaneously  in  table  3.  The  values  of  ny 
were  selected  by  extensive  experiments  on  engine  simula¬ 
tion  models  and  were  also  based  on  experience  of  gas 


Figure  2.  Unsupervised  plant,  i.e.,  no  disabling  of 
controllable  events. 
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turbine  engine  operation  and  maintenance.  The  dump 
state  and  any  transitions  to  the  dumped  state  are  not 
shown  in  table  3.  The  elements  of  the  characteristic 
vector  (see  Definition  5)  were  chosen  as  signed  real 


Table  1.  Plant  automation  states. 

State  Description 

1  Safe  in  base 

2  Mission  executing  —  two  good  engines 

3  One  engine  unhealthy  during  mission  execution 

4  Mission  executing  —  one  good  and  one  unhealthy  engine 

5  Both  engines  unhealthy  during  mission  execution 

6  One  engine  good  and  one  engine  inoperable 

7  Mission  execution  with  two  unhealthy  engines 

8  Mission  execution  with  only  one  good  engine 

9  One  engine  unhealthy  and  one  engine  inoperable 

10  Mission  execution  with  only  one  unhealthy  engine 

11  Mission  aborted/not  completed  (Bad  Marked  State) 

12  Mission  successful  (Good  Marked  State) 

13  Aircraft  destroyed  (Bad  Marked  State) 


Table  2.  Plant  event  alphabet. 


Event  Controllable 

Event  description  event 


s 

Start  and  take-off 

V 

b 

A  good  engine  becoming  unhealthy 

t 

An  unhealthy  engine  becoming  inoperable 

V 

A  good  engine  becoming  inoperable 

k 

Keep  engine(s)  running 

V 

a 

Mission  abortion 

V 

f 

Mission  completion 

d 

Destroyed  aircraft 

1 

Landing 

V 

weights  based  on  the  perception  of  each  marked  state’s 
role  on  the  engine  performance. 

The  characteristic  (/)  values  of  the  13  states 
are  assigned  as:  X  =  [0  000000000  —0.05  +0.25 
— 1.0]7.  These  parameters  are  selected  by  the  designer 
based  on  his/her  perception  of  each  marked  state’s 
role  in  the  system  performance.  As  the  states  1  to  10 
in  in  table  1  are  not  marked,  the  first  10  elements  of 
the  characteristic  vector  X  are  zeros.  The  implication 
is  that  event  strings  terminating  at  states  1  to  10 
have  no  bearing  on  the  system  performance  and 
hence  have  zero  measure.  The  state  12  is  a  good 
marked  state  having  a  positive  /  value  and  the  bad 
marked  states  11  and  13  have  negative  x  values. 
Therefore,  event  strings  terminating  at  state  12  have 
positive  measure  and  those  terminating  at  states  11 
and  13  have  negative  measure. 

Four  supervisory  controllers  were  designed  indepen¬ 
dently  using  a  graphical  interactive  package  (Wang 
et  al.  2003)  based  on  the  following  specifications. 

(1)  Specification  #1:  At  least  one  of  the  two  engines 
must  be  in  good  condition  for  mission  continuation. 

(2)  Specification  #2:  None  of  the  two  engines  must  be  in 
inoperable  condition  for  mission  continuation. 

(3)  Specification  #3:  Both  engines  must  be  in  good  con¬ 
dition  for  mission  continuation. 

(4)  Optimal  Control:  The  control  policy  is  optimized 
following  the  two-step  procedure  in  §  6.2. 

The  supervised  plant  automata  under  specifications 
#1,  #2,  #3  and  optimal  control  are  displayed  in 
figures  3,  4,  5  and  6,  respectively,  where  dashed  lines 
indicate  disabled  controllable  events.  Notice  that  none 
of  the  controllable  events  are  disabled  in  the  unsuper¬ 
vised  plant  (see  figure  2)  and  the  four  supervisors 
disable  different  sets  of  controllable  events,  as  seen  in 
figures  3-6. 


Table  3.  State  transition  and  event  cost  matrix. 


s 

b 

t 

V 

k 

a 

/ 

d 

/ 

1  0.50(2) 

0.02(1) 

2 

0.05(3) 

0.01(6) 

0.80(12) 

0.10(13) 

3 

0.45(4) 

0.45(11) 

4 

0.12(5) 

0.16(6) 

0.10(9) 

0.50(12) 

0.12(13) 

5 

0.45(7) 

0.45(11) 

6 

0.45(8) 

0.45(11) 

7 

0.25(9) 

0.50(12) 

0.20(13) 

8 

0.20(9) 

0.01(13) 

0.3(12) 

0.4(13) 

9 

0.45(10) 

0.45(11) 

10 

0.35(13) 

0.20(12) 

0.40(13) 

11 

0.95(1) 

12 

0.95(1) 

13 
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Figure  6.  Optimally  supervised  plant,  i.e.,  with  best 
performance. 


The  performance  measure  (i\  (i.e.,  with  the  initial 
state  1)  of  the  unsupervised  (i.e.,  no  disabling  of  control 
events)  plant  is  0.0823  and  for  three  supervised  plants 
under  specifications  #1,  #2,  #3  and  the  optimally 


supervised  plant  are  evaluated  to  be:  0.0807,  0.0822, 
0.0840  and  0.0850,  respectively.  Therefore,  the  perfor¬ 
mance  of  the  supervised  plant  under  specifications  #1, 
#2  and  #3  is  inferior,  similar,  and  superior,  respectively, 
to  that  of  the  unsupervised  plant.  As  expected,  the  opti¬ 
mal  supervisor  has  better  performance  than  that  of 
Supervisor  #3.  Notice  that  Supervisor  #3  does  disable 
the  controllable  event  k  from  the  state  3  to  state  4  and 
the  optimal  supervisor  does  not.  That  is,  the  optimal 
supervisor  allows  continuing  operation  of  an  unhealthy 
engine  while  the  remaining  engine  is  in  good  condition. 

8.  Summary,  conclusions,  and  future  research 

This  paper  reviews  the  concept,  formulation  and  valida¬ 
tion  of  a  signed  real  measure  for  regular  languages  and 
their  sublanguages  based  on  the  principles  of  automata 
theory  and  real  analysis.  While  the  domain  of  the  mea¬ 
sure  /r,  i.e.,  2^g‘\  is  partially  ordered,  its  range,  which  is 
a  subset  of  R  =  (— oo,  oo),  becomes  totally  ordered.  As  a 
result,  the  relative  performance  of  different  supervisors 
can  be  quantitatively  evaluated  in  terms  of  the  real 
signed  measure  of  the  supervised  sublanguages. 
Positive  weights  are  assigned  to  good  marked  states 
and  negative  weights  to  bad  marked  states  so  that  a  con¬ 
trollable  supervisor  is  rewarded  (penalized)  for  deleting 
strings  terminating  at  bad  (good)  marked  states.  In 
order  to  evaluate  and  compare  the  performance  of  dif¬ 
ferent  supervisors  a  common  quantitative  tool  is 
required.  To  this  effect,  the  proposed  procedure  com¬ 
putes  the  measure  of  the  supervised  sublanguage  gener¬ 
ated  by  a  supervisor  using  the  event  cost  and 
characteristic  function  assigned  for  the  unsupervised 
plant.  Cost  assignment  to  each  event  based  on  the 
state,  where  it  is  generated,  has  been  shown  similar  to 
the  conditional  probability  of  the  event.  On  the  other 
hand,  the  characteristic  function  is  chosen  based  on 
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the  designer’s  perception  of  the  individual  state’s  impact 
on  the  system  performance.  Two  techniques  are  pre¬ 
sented  to  compute  the  language  measure  for  a  DFSA. 
One  of  them  yields  a  closed  form  solution  that  is 
obtained  as  the  unique  solution  of  a  set  of  linearly  inde¬ 
pendent  algebraic  equations.  The  other  is  based  on  a 
recursive  procedure.  The  computational  complexity  of 
both  language  measure  algorithms  is  identical  and  is 
of  polynomial  order  in  the  number  of  states  of  the 
DFSA.  As  such  it  is  relatively  straight-forward  to 
develop  software  analysis  tools  in  standard  languages 
such  as  Matlab,  C,  and  Java  (Wang  et  al.  2003). 

8.1.  Recommendations  for  future  research 

Optimal  discrete-event  supervisory  control  can  be 
enhanced  through  appropriate  usage  of  the  language 
measure.  For  example,  in  the  current  configuration  of 
the  optimal  control  algorithm  (Ray  et  al.  2004),  if 
there  is  no  state  qj  such  that  /d1  <  0,  then  the  plant 
performance  cannot  be  improved  by  event  disabling 
and  the  null  controller  S°  (i.e.,  no  disabled  event)  is 
the  optimal  controller  for  the  given  plant.  This  restric¬ 
tion  can  be  lifted  through  an  appropriate  performance 
index  that  would  be  a  function  of  the  language  measure 
fi  but  not  identically  equal  to  fi  as  reported  in  Ray  et  al. 
(2004).  Work  in  this  direction  is  in  progress  and  is 
expected  to  be  reported  in  a  forthcoming  publication. 

Synthesis  of  supervisory  control  systems  may  become 
a  significant  challenge  if  some  of  the  events  are  delayed, 
intermittent,  or  not  observable  at  all,  possibly  due  to 
sensor  faults  or  malfunctions  in  network  communica¬ 
tion  links.  In  that  case,  the  control  algorithms  may 
turn  out  to  be  computationally  very  complex  because 
of  delayed  or  lost  information  on  the  plant  dynamics. 
Future  work  in  this  direction  should  involve  research 
on  construction  of  language  measures  under  partial 
observation  (Chattopadhyay  and  Ray  2004)  and 
synthesis  of  optimal  control  policies  under  partial  obser¬ 
vation  to  mitigate  the  detrimental  effects  of  loss  of 
observability.  The  latter  research  could  be  an  extension 
of  the  earlier  work  on  optimal  control  under  full 
observation  (Ray  et  al.  2004). 

It  would  be  a  challenging  task  to  extend  the  concept 
of  (regular)  language  measure  for  languages  higher  up 
in  the  Chomsky  Hierarchy  (Martin  1997)  such  as  con¬ 
text  free  and  context  sensitive  languages.  This  extension 
would  lead  to  controller  synthesis  when  the  plant 
dynamics  is  modelled  by  non-regular  languages  such 
as  the  Petri-Net.  The  research  thrust  should  focus 
on  retaining  the  polynomial  order  of  computational 
complexity. 

Another  critical  issue  is  how  to  extend  the  language 
measure  for  timed  automaton,  especially  if  the  events 
are  observed  with  varying  delays  at  different  states. 


Another  research  topic  that  may  also  be  worth  investi¬ 
gating  is:  how  to  extend  the  field  GF( 2),  over  which 
the  vector  space  of  languages  has  been  defined,  to 
richer  fields  like  the  set  of  real  numbers. 

Other  areas  of  future  research  include  applications  of 
the  language  measure  in  anomaly  detection,  model 
identification,  model  order  reduction,  and  analysis  and 
synthesis  of  interfaces  between  the  continuously- 
varying  and  discrete-event  spaces  in  the  language- 
measure  setting. 

Appendices  A:  Measure  theory 

This  appendix  introduces  the  concepts  of  standard 
measure-theoretic  quantities  that  are  used  to  establish 
the  language  measure  in  the  main  body  of  this  paper. 

Definition  A.l:  A  a-algebra  M  of  a  nonempty 
language  L(G ,)  c  E*  is  a  collection  of  subsets  of  L(G,) 
which  satisfies  the  following  three  conditions. 

(i)  L(Gj)  e  M\ 

(ii)  If  KeM,  then  (L(G, )  -  K)  e  M; 

(iii)  eM  if  Kj  e  M  Vy. 

Definition  A.2:  An  at  most  countable  collection  [Lk] 
of  members  of  a  a-algebra  M  is  a  partition  of  a 
member  L  e  M  if  L  —  14  L*  and  Lk  fl  Lj  —  0  Wk  j. 

Definition  A.3:  Let  M  be  a  er-algebra  of  L(G,).  Then, 
the  set  function  /x:  M  — >  R  =  (— oo,  +  oo),  is  called  a 
signed  real  measure  if  the  following  two  conditions  are 
satisfied  (Rudin  1987): 

(i)  4(0)  =  0; 

(ii)  /x(Uy4j  Lj)  —  ll(Lj)  for  every  partition  {Lj}  on 
any  member  L  e  M. 

Note  that,  unlike  a  positive  measure  (e.g.,  the  Lebesgue 
measure),  /x  is  finite  such  that  the  series  in  part  (ii)  of 
Definition  A.3  converges  absolutely  in  R  and  the  result 
is  independent  of  any  permutation  of  the  terms  under 
union. 

Definition  A.4:  Relative  to  the  signed  real  measure  /x, 
a  sublanguage  L  e  M  is  defined  to  be 

(i)  null,  denoted  as  L  =  0  ,  if  /x(L  n  J )  =  0,  V/  e  M\ 

(ii)  positive,  denoted  as  L  >  0,  if  L  f  0  and 
n(L  n  /)  >  0,  V/  e  M- 

(iii)  negative,  denoted  as  L,  if  L  f  0  and  /x(L  fl  /)  <  0, 
V/  e  M. 

Definition  A.5:  Total  variation  |/x|  on  a  a-algebra  M  is 
defined  as 

\ll\(L)  —  sup  ^  \fi(Lk)\  (52) 

k 


966 


A.  Ray 


VP  e  M  where  the  supremum  is  taken  over  all  partitions 
{Lk}  of  L. 

Proposition  A.l:  Total  variation  measure  \p\  of  any 
regular  language  L  is  non-negative  and  finite  i.e.,  \p\  (P)  e 
[0,  oo).  The  proof  follows  from  standard  theorems  on 
complex  measures  ( Rudin  1987). 

Total  variation  can  be,  in  general,  defined  for  complex 
measures  (Rudin  1987)  but  it  is  restricted  to  a  signed 
real  measure  in  this  paper.  The  total  variation  of  a 
real  signed  measure  p,  can  be  represented  as  \p\  — 
p+  +  p~  where  p+  and  p~  are  called  positive  and  nega¬ 
tive  variations  of  p  and  are  defined  as 

d+  =  j(lRl  +  d)  and  p~  =  \(\p\  -  p).  (53) 

Both  p+  and  p~  are  positive  measures  on  M.  It  also 
follows  from  the  above  equation  that  p  =  p+  —  p~ . 
This  representation  of  p  as  the  difference  of  positive 
measure  p+  and  p~  is  known  as  the  Jordan 
Decomposition  of  p  (Rudin  1987). 

Proposition  A. 2:  Every  sublanguage  L  e  M  can  be 
partitioned  as  L  =  P°  U  L+  U  L~  where  the  mutually 
exclusive  sublanguages  L° ,  L+  and  L~  are  called  null, 
positive,  and  negative,  respectively,  relative  to  a  signed 
real  measure  p. 

Proof:  The  proof  is  based  on  the  Hahn  Decomposition 
Theorem  (Rudin  1987).  □ 

As  a  consequence  of  the  above  result,  the  following  rela¬ 
tions  hold  VP  e  M  for  positive  and  negative  variations: 

p+(L)  =  /x(Lp|L+)  and  p~(L)  =  -/r(z,p|Zr). 

(54) 

Appendix  B:  Convergence  of  the  language  measure 

This  appendix  establishes  necessary  and  sufficient  condi¬ 
tions  for  finiteness  of  the  measure  p,  based  on  certain 
properties  of  non-negative  matrices,  which  are  stated 
without  proof.  Details  of  these  results  are  available  in 
(Senata  1973,  Plemmons  and  Berman  1979). 

Definition  B.l:  Let  A  and  B  be  real  square  matrices  of 
the  same  order  n.  Then,  the  notations  for  inequalities  are 
as  follows: 

A  >  B  if  an  >  bn,  Wi,i 
A  >  B  if  A  >B,  A  fB 
4  >  B  if  aj  >  bjj,  Vi,j. 

If  the  matrix  A  satisfies  the  condition  A  >  0,  i.e.  the 
null  matrix,  then  A  is  called  a  non-negative  matrix 


and  if  the  condition  A  yp  0  is  satisfied,  then  it  is  called 
a  positive  matrix. 

Definition  B.2:  A  square  matrix  A  of  order  n  is  cogra¬ 
dient  to  a  matrix  E  if  PAP T  —  E  for  a  permutation 
matrix  P;  and  A  is  called  reducible  if  A  is  cogradient  to 


where  B  and  C  are  square  matrices,  or  if  n  =  1  and 
A  —  0.  Otherwise,  A  is  irreducible. 

It  follows  from  the  above  definition  that  a  positive 
matrix  is  always  irreducible. 

Proposition  B.l:  A  non-negative  matrix  A  is  irreducible 
if  and  only  if,  for  every  ( i,j )  there  exists  a  natural  number 
k  such  that  d)^  >  0,  where  a^]  denotes  the  ( i,j)th  element 
of  Ak . 

Proposition  B.2:  If  A  >  0  is  irreducible  and  B  >  0,  then 
A  +  B  is  irreducible. 

Another  characterization  of  irreducibility  of  a 
non-negative  square  matrix  has  a  graph-theoretic 
interpretation.  This  relationship  can  help  to  determine 
under  what  conditions  a  given  finite  state  automaton 
G,  which  represents  the  supervised  or  unsupervised 
plant  model  is  irreducible  by  looking  at  connectivity  of 
its  states.  The  following  definitions  are  needed  to 
arrive  at  this  conclusion. 

Definition  B.3:  The  associated  directed  graph,  G(A)  of 
a  square  matrix  A  of  order  n,  consists  of  n  vertices 
P] ,  P2, . . . ,  P„  where  an  edge  leads  from  P,  to  P/  if  and 
only  if  Uy  f  0. 

Definition  B.4:  A  directed  graph  G  is  strongly  con¬ 
nected  if  for  any  ordered  pair  (P,,  P;)  of  vertices  of  G, 
there  exists  a  sequence  of  edges  which  leads  from  P,- 
to  Pj. 

Proposition  B.3:  Given  a  matrix  A,  it  is  irreducible  if 
and  only  if  G(A)  is  strongly  connected. 

If  A  is  a  non-negative  square  matrix,  then  the  following 
relationship  holds  between  the  spectral  radius  (i.e., 
maximum  absolute  eigenvalue)  p  of  non-negative 
matrices. 

Proposition  B.4:  If0<A<B  and  A  +  B  is  irreducible 
then  p{A)  <  p(B). 

Definition  B.5:  A  square  matrix  S  of  order  n  is  called 
(row)  stochastic  if  it  satisfies 

n 

Sy>  0,  y^.Sjj  —  1,  1  <  i  <  n.  (55) 
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Proposition  B.5:  The  maximum  eigenvalue  of  a  stochas¬ 
tic  matrix  S  is  one,  i.e.  p(S )  =  1 .  A  non-negative  matrix  A 
is  stochastic  if  and  only  if  e  is  an  eigenvector  of  A  corre¬ 
sponding  to  the  eigenvalue  one,  where  e  is  the  vector  all  of 
whose  entries  are  ecpial  to  one. 

In  order  to  show  that  (/  —  II)-1  is  invertible  it  suffices  to 
show  that  p(II)  <  1. 

Theorem  B.l:  If  p(\Y)  <  1  then  there  exists  at  least  one 
i,  1  <  i  <  n,  such  that  f2j=\  71 ij  <  L 

Proof:  Proof  follows  from  the  fact  that  if  Jfj=i  71 ij  —  1 
Vi,  then  II  would  be  a  stochastic  matrix  by  Definition 
B.5.  Hence,  by  Proposition,  B.5  p(II)  =  1  =>■  (/  —  II)-1 
is  not  invertible.  □ 

Theorem  B.2:  If  i  71  ij  <  1  Vi,  1  <  i  <  n,  then  p(IT). 

Proof:  Let  0,  =  (1  —  n ij )/n  >  0-  Let  S  be  a  matrix 
of  order  n  which  is  defined  in  the  following  manner 

Sjj  =  0,  +  TCj,  VI  <i,j<n. 

It  is  clear  that  5  0  and  hence  S  is  irreducible.  Also  5 

is  a  stochastic  matrix  and  by  Proposition  B.5,  the  spec¬ 
tral  radius  p(S)  —  1 .  Since  0  <  IT  <  and  IT  +  S  is  irredu¬ 
cible  by  Proposition  B.2,  it  follows  that  p(IT)  <  p(S)  —  1 
from  Proposition  B.4.  □ 

The  above  sufficiency  condition  is  more  strict  than  the 
necessary  condition  required  in  Theorem  B.l. 
However,  the  necessary  condition  is  not  sufficient  as 
seen  from  the  following  example. 


/  0.2 

0 

0.8 

0\ 

0 

0.2 

0.3 

0.5 

0.5 

0 

0.5 

0 

\01 

0.2 

0.4 

0/ 

This  matrix  II  satisfies  conditions  as  required  in 
Theorem  B.l,  but  p(II)  =  1.  It  is  possible  to  relax  the 
strict  inequality  ffj=\  77 j  <  1  Vi,  1  <i<n  in  Theorem 
B.2,  but  with  additional  conditions  on  structure  of  II. 
For  example,  under  such  relaxed  conditions,  if  IT  +  5 
is  irreducible,  then  still  p(II)  <  1.  This  follows  from 
the  fact  that  application  of  Proposition  B.4.  only 
requires  irreducibility  of  IT  +  S.  In  order  to  determine 
the  irreducibility  of  a  matrix,  the  graph-theoretic 
interpretation,  described  earlier,  can  be  a  useful  tool. 
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Supervisory  control  theory  for  discrete  event  systems,  introduced  by  Ramadge  and  Wonham, 
is  based  on  a  non-probabilistic  formal  language  framework.  However,  models  for  physical 
processes  inherently  involve  modelling  errors  and  noise-corrupted  observations,  implying 
that  any  practical  finite-state  approximation  would  require  consideration  of  event  occurrence 
probabilities.  Building  on  the  concept  of  signed  real  measure  of  regular  languages,  this  paper 
formulates  a  comprehensive  theory  for  optimal  control  of  finite-state  probabilistic  processes. 
It  is  shown  that  the  resulting  discrete-event  supervisor  is  optimal  in  the  sense  of  elementwise 
maximizing  the  renormalized  langauge  measure  vector  for  the  controlled  plant  behaviour  and 
is  efficiently  computable.  The  theoretical  results  are  validated  through  several  examples  includ¬ 
ing  the  simulation  of  an  engineering  problem. 


1.  Introduction 

Supervisory  control  theory  (SCT)  of  discrete-event 
systems  (DES),  pioneered  by  Ramadge  and 
Wonham  (1987),  models  a  physical  or  human-engi¬ 
neered  process  as  a  finite-state  language  generator  and 
constructs  a  supervisor  that  attempts  to  constrain  the 
“supervized”  plant  behaviour  within  a  specification  lan¬ 
guage.  The  original  theory  is  based  on  a  deterministic 
language  framework.  Although  allowing  non-determin¬ 
ism  in  the  sense  that  more  than  one  continuation  of  a 
generated  event  trace  (i.e.,  a  string)  is  possible,  no 
attempt  is  made  to  quantify  this  randomness. 
As  Wonham  himself  observes  in  Lawford  and 
Wonham  (1993),  “the  choice  of  a  possible  continuation 
of  a  string  is  made  by  some  internal  structure  unmodeled 
by  the  systems  designer”.  The  notion  of  probabilistic 
languages  in  the  context  of  studying  qualitative  stochas¬ 
tic  behaviour  of  discrete-event  systems  first  appears 
in  Garg  (1992a,  b),  where  the  concept  of  p-languages 
(‘p’  implying  probabilistic)  is  introduced  and  an  algebra 
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is  developed  to  model  probabilistic  languages  based  on 
concurrency  (Milner  1989).  A  regular  p-language  is 
essentially  a  set  of  prefix-closed  traces  of  events, 
generated  by  a  finite-state  automaton  with  probabilities 
associated  with  the  transitions.  A  p-language-theoretic 
model  differs  in  several  important  aspects  from 
other  discrete-event  models  of  stochastic  analysis 
such  as  Markov  chains  (Cassandras  and  Lafortune 
1999),  stochastic  Petri  nets  (Molloy  1982,  Chung  et  al. 
1994),  probabilistic  automata  (Rabin  1963,  Paz  1971, 
Doberkat  1981),  and  fuzzy  models  (Lee  and 
Zadeh  1969).  Garg  et  al.  (1999)  and  Kumar  and  Garg 
(2001)  provide  a  brief  comparison  of  the  p-language- 
theoretic  modelling  paradigm  with  the  above-mentioned 
theories. 

Lawford  and  Wonham  (1993)  have  attempted  to 
extend  discrete-event  (SCT)  to  plants  modelled  by 
p-languages,  where  a  formal  statement  of  the  probabilis¬ 
tic  supervisory  control  problem  (PSCP)  first  appears  and 
the  notion  of  probabilistic  supervision  is  introduced  by 
random  disabling  of  controllable  events.  The  key  differ¬ 
ence  from  other  stochastic  supervision  approaches  (e.g., 
Mortzavian  1993)  lies  in  the  fact  that  the  computed 
probabilistic  supervisor  is  not  allowed  to  change  the 
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underlying  plant  dynamics  in  the  following  sense: 
“The  probabilistic  effect  of  random  disablement  is  deter¬ 
mined  entirely  by  the  plant”.  The  control  objective  is 
specihed  as  a  p-language  and  necessary  and  su  ITicicnt 
conditions  are  derived  for  existence  of  a  probabilistic 
supervisor  that  attempts  to  restrict  the  plant  language 
within  the  control  specihcation  in  a  probabilistic  sense. 
The  theory  of  supervision  of  p-languages  is  further 
developed  by  Kumar  and  Garg  (2001),  where  the  control 
objective  is  specified  as  upper  and  lower  bound 
constraints.  The  upper  bound  is  a  non-probabilistic 
language  that  serves  as  a  legality  constraint,  while  the 
lower  bound  is  a  p-language.  This  relatively  relaxed 
approach  to  control  objective  specification  allows  for  a 
non-probabilistic  supervisor  that  attempts  to  cut  down 
illegal  event  traces,  while  ensuring  that  legal  traces 
occur  with  probabilities  greater  than  or  equal  to  that 
specihed  by  the  lower  bound.  Intuitively,  the  designed 
supervisor  stops  “bad”  strings  from  occurring  while 
guaranteeing  that  “good”  strings  occur  with  some  mini¬ 
mum  pre-set  frequency.  However,  construction  of  such 
a  control  objective  specihcation  may  not  be  possible  in 
many  applications  (e.g.,  battlefield  command,  control, 
communications,  and  intelligence  (C3I)  (Phoha  et  al. 
2002)),  especially  if  the  decisions  are  to  made  in  real 
time.  For  the  theory  to  be  useful  in  practice,  one 
must  generate  the  specihcation  from  the  definition  of 
the  physical  problem  at  hand.  Given  that  one  has 
to  come  up  with  a  non-probabilistic  language  to  serve 
as  the  upper  legality  constraint  and  a  probabilistic 
language  to  serve  as  the  lower  bound,  this  goal 
may  not  always  be  achievable.  The  situation  becomes 
worse  for  non-stationary  stochastic  environments, 
where  the  control  specihcations  may  have  to  be  updated 
online. 

A  signihcantly  simplihed  approach  to  the  above  pro¬ 
blem  is  reported  by  Ray  (2005)  and  Ray  et  al.  (2005), 
where  the  control  objective  is  specihed  as  characteristic 
weights  on  the  states  of  the  plant  automaton.  These 
weights  are  normalized  in  the  interval  [—1,1]  with  posi¬ 
tive  weights  assigned  to  good  states  and  negative  weights 
to  bad  states.  A  signed  real  measure  of  regular  languages 
(of  event  traces)  is  defined  as  a  function  of  the  character¬ 
istic  weights  and  the  state  transition  probabilities;  and 
supervisory  control  laws  are  synthesized  by  elementwise 
maximizing  the  language  measure  vector  (Ray  et  al. 
2004,  2005).  Intuitively,  the  supervisor  ensures  that  the 
generated  event  traces  cause  the  plant  to  visit  the 
“good”  states  while  attempting  to  avoid  the  “bad” 
states  in  a  probabilistic  sense.  As  mentioned  earlier, 
Kumar  and  Garg’s  work  on  supervisory  control  of 
probabilistic  automata  (Kumar  and  Garg  2001)  also 
has  a  notion  of  “good”  and  “bad”  strings.  However, 
the  classification  is  strictly  binary;  the  theory  has  no 
way  of  saying  if  one  “good”  string  is  “better”  than 


another  “good”  string  and  vice  versa.  This  implies  that 
the  supervisor  must  eliminate  all  bad  strings  and  hence 
may  not  be  optimal,  or  fail  to  exist  if  the  conditions 
defined  in  Kumar  and  Garg  (2001)  are  not  satisfied. 
In  contrast,  in  the  measure-theoretic  approach 
(Ray  2005),  the  event  traces  are  less  or  more  desirable 
in  a  continuous  scale  with  the  supervisor  optimizing 
the  controlled  plant  behaviour  to  ensure  that  the 
“most”  desirable  strings  occur  “most”  often.  This  has 
an  immediate  advantage  that  the  problem  of  existence 
disappears;  the  optimal  supervisor  always  exists  and 
can  be  computed  effectively  with  polynomial  complex¬ 
ity.  The  latter  approach  is,  in  one  sense,  closer  to 
Markov  chain  modelling  since  the  control  specification 
is  state-based.  However,  as  shown  by  Ray  (2005), 
this  does  not  restrict  the  modeling  power  of  the  techni¬ 
que.  It  is  shown  in  Kumar  and  Garg  (2001)  that, 
in  general,  maximally  permissive  supervisors  are 
non-unique.  For  the  measure-theoretic  approach,  how¬ 
ever,  the  optimization  is  shown  to  yield  unique  maximal 
permissiveness  among  all  optimal  supervisors  (Ray  et  al. 
2004,  2005). 

Optimal  control  in  the  context  of  discrete  event 
dynamic  systems  has  been  addressed  earlier  by  several 
investigators  as  cited  in  (Ray  et  al.  2004).  For  example, 
Sengupta  and  Lafortune  (1998)  have  analysed 
non-probabilistic  DES  with  assigned  event  and  control 
costs;  the  optimal  supervisor  is  computed  in  the  frame¬ 
work  of  dynamic  programming  (DP)  with  two  critical 
assumptions  to  guarantee  polynomial  complexity  of 
the  solution:  all  costs  are  strictly  positive  and  there  is 
only  one  marked  state  (Sengupta  and  Lafortune  1998, 
p.  34).  The  work  reported  in  Ray  (2005)  and  Ray 
et  al.  (2005)  is  different  in  the  sense  that  the  latter 
deals  with  probabilistic  automata  and  the  optimization, 
even  in  the  completely  general  case,  has  guaranteed 
polynomial  complexity  of  0(n3),  where  n  is  the  number 
of  states  in  the  unsupervised  plant  model.  The  mea¬ 
sure-theoretic  approach  was  originally  reported  for  a 
restricted  class  of  terminating  p-languages  Ray  (2005) 
and  Ray  et  al.  (2005);  and  this  restriction  has  been  elimi¬ 
nated  in  a  subsequent  publication  (Chattopadhyay  and 
Ray  2006a). 

The  notion  of  terminating  and  non-terminating  auto¬ 
mata  is  originally  due  to  Garg  (19992a,  b).  A  probabil¬ 
istic  automaton  is  terminating  if  there  exist  states  at 
which  the  sum  of  the  probabilities  of  all  defined  events 
is  strictly  less  than  1 .  The  interpretation  is  that  the  differ¬ 
ence  of  the  sum  from  1  is  the  probability  that  the  plant 
terminates  operation  at  that  particular  state.  It  is  shown 
in  Ray  (2005)  that  the  language  measure  vector  can  be 
expressed  as  [D  —  n]~ 1  x  where  II  is  the  transition  prob¬ 
ability  matrix  and  x  is  the  characteristic  vector,  where 
Ttij  is  the  probability  of  transition  from  the  zth  state  to 
the  /th  state  and  yA  is  the  characteristic  weight  of  the 
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state  i).  A  sufficient  condition  for  the  inverse  of  [l  —  II] 
to  exist  is  that  ity  <1  e  i,  i.e.,  the  plant  has  a  strictly 
non-zero  probability  of  termination  from  each  state. 

This  paper  eliminates  the  above  restrictive  assumption 
by  adopting  the  recently  reported  renormalized  measure 
of  regular  languages  (Chattopadhyay  and  Ray  2006a) 
as  the  performance  index.  It  also  extends  the  measure- 
theoretic  concept  for  optimal  control  of  terminating 
plants  (Ray  et  al.  2004)  to  non-terminating  plant 
models,  which  requires  a  minor  modification  of  the 
control  philosophy  as  explained  below. 

Supervisors  in  the  SCT  paradigm  are  allowed  to  affect 
the  underlying  plant  behaviour  by  selectively  disabling 
controllable  events  (Ramadge  and  Wonham  1987). 
In  case  of  terminating  p-languages,  a  similar  approach 
suffices;  the  supervisor  selectively  nulls  the  occurrence 
probability  of  controllable  events  to  achieve  the  desired 
control  objective.  However,  the  non-terminating  case 
poses  a  problem  since  any  such  disabling  action  converts 
the  system  to  a  terminating  p-language  (i.e.,  the  prob¬ 
abilities  of  the  events  defined  at  a  state  fail  to  add  up 
to  1).  The  solution  (Kumar  and  Garg  2001)  is  to  propor¬ 
tionately  increase  the  probabilities  of  the  remaining 
enabled  events  at  the  state  at  which  event  disabling  is 
undertaken.  An  alternative  approach  is  proposed  in 
this  paper,  where  each  disabled  event  creates  a  self 
loop  at  the  state  (at  which  the  event  was  generated) 
with  occurrence  probability  of  the  original  transition. 

The  paper  is  organized  in  six  sections  and  an 
appendix.  Section  2  lays  down  the  basic  framework  of 
the  analysis  and  briefly  reviews  the  original  notion  of 
language  measure  (Ray  2005)  and  its  renormalization 
(Chattopadhyay  and  Ray  2006a).  Section  3  formulates 
the  optimal  control  problem  based  on  the  concept  of 
renormalized  measure  and  presents  the  key  results. 
Section  4  presents  a  solution  of  the  optimal  control 
problem  and  derives  the  necessary  algorithms  for  its 
implementation.  Section  5  presents  an  engineering 
example,  where  the  optimal  supervisor  is  designed  for 
a  three-processor  message  decoding  system.  The 
paper  is  summarized  and  concluded  in  §6  along  with 
recommendations  for  future  research.  Appendix  G 
establishes  bounds  on  the  derivatives  of  the 
renormalized  measure  that  is  necessary  for  formulation 
of  the  optimal  control  law  in  §3. 


2.  Preliminary  concepts 


2.1  Brief  review  of  language  measure 

Let  the  plant  behaviour  be  modelled  as  a  deterministic 
finite  state  automaton  (DFSA)  as  G,  =  ((7,  £,5,  <?,■,  Q,„), 
where  Q  is  the  finite  set  of  states  with  \Q\  —  n,  and 
cpeQ  is  the  initial  state;  £  is  the  (finite)  alphabet  of 
events  with  |£|  =  m;  the  Kleene  closure  of  £  is  denoted 
as  £*  that  is  the  set  of  all  finite-length  strings  of  events 
including  the  empty  string  e;  the  (possibly  partial)  func¬ 
tion  8 :  Q  x  £  — >  Q  represents  state  transitions  and 
8*:  Q  x  £*  — >  Q  is  an  extension  of  <5;  and  Qm  c  Q  is 
the  set  of  marked  (i.e.,  accepted)  states. 

Definition  1:  The  language  L(G,)  generated  by  the 
DFSA  Gi  is  defined  as  L(G,)  =  {s  e  £*  |  8*(q,,s)  e  Qj. 

Definition  2:  The  marked  language  Lm(G,)  by  the 
DFSA  G/  is  defined  as  Lm(G,)  =  £*|  8*{qt,s )  e  Q,,,}. 

The  language  L(G,)  of  the  DFSA  G,  is  partitioned  as 
the  non-marked  and  the  marked  languages, 
L"{Gj)  =  L(Gi)  —  Lm(Gi )  and  L,„(G,),  consisting  of  event 
strings  that,  starting  from  cp  e  Q,  terminate  at  one  of 
the  non-marked  states  in  Q  —  Qm  and  one  of 
the  marked  states  in  Qm,  respectively.  The  set  Qm  is 
partitioned  into  Q+  and  Q~  where  Q+  contains  all 
good  marked  states  that  one  may  desire  to  terminate 
on,  and  Q~  contains  all  bad  marked  states  that  one 
would  attempt  to  avoid  terminating  on,  although  it 
may  not  always  be  possible  to  bypass  a  bad  state 
before  reaching  a  good  state.  The  marked  language 
Lm(Gi )  is  further  partitioned  into  L+(Gj)  and  L~(G;) 
consisting  of  good  and  bad  strings  that,  starting  from 
cp,  terminate  on  Q+  and  Q~,  respectively. 

A  signed  real  measure  p:  2E*  ->  R  =  (— oo,  oo)  is 
constructed  for  quantitative  evaluation  of  every  event 
string  5  e  £*.  The  language  L(G,)  is  decomposed  into 
null,  i.e.,  L°(Gj),  positive,  i.e.,  L+(G,j,  and  negative, 
i.e.,  L~(Gi)  sublanguages. 

Definition  3:  The  language  of  all  strings  that,  starting 
at  a  state  qt  e  Q,  terminates  on  a  state  qj  e  Q,  is  denoted 
as  L(cp,  qj).  That  is, 

L(qh  cp)  /-( G,j  :  8fqt,  s)  -  q,}.  (1) 


Definition  4:  The  characteristic  function  that  assigns  a 
signed  real  weight  to  each  state  qi,  i  —  1,2, . . .  ,n,  is 
defined  as:  x:  Q  [—  T  1]  such  that 


This  section  briefly  reviews  the  concept  of  signed  real 
measure  of  regular  languages  Ray  (2005)  and  Ray 
et  al.  (2005)  followed  by  a  review  of  the  notion  of 
renormalized  measure  and  the  pertinent  notations  used 
in  the  sequel. 


[-1,0)  if  qjeQm 


xfii)  e 


{0}  if  qjtQm 


(0,1]  if  cpeQ+ 
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Definition  5:  The  event  cost  is  conditioned  on  a  DFSA 
state  at  which  the  event  is  generated,  and  is  defined  as 
it :  E*  x  Q  — >  [0,  1]  such  that  Vqj  e  Q,  Vak  e  E, 
Vs  e  E\ 

1.  7T[ak,  qj  =  7TJk  e  [0, 1);  J2kKjk  <  1; 

2.  7r[cr,  qj\  —  0  if  S(qj,a )  is  undefined;  ir[e,  qj]  —  1; 

3.  Tt[aks,  q,\  —  n[ak,  qj  jt[s,  S(qj,  ok)\. 

The  event  cost  matrix,  denoted  as  Il-matrix,  is 
defined  as 


n  = 


1 


Xlm 

rtnm 


An  application  of  the  induction  principle  to  part  (3)  in 
Definition  5  shows  ix[st,  qj  —  tt[s,  qjit[l,  8*(q/,  s)]. 
The  condition  E^if*  <  1  provides  a  sufficient  condition 
for  the  existence  of  the  real  signed  measure  as  discussed 
in  Ray  (2005)  along  with  additional  comments  on  the 
physical  interpretation  of  the  event  cost. 

Now  let  us  define  the  measure  of  a  sublanguage  of  the 
plant  language  L{Gj)  in  terms  of  the  signed  characteristic 
function  x  and  the  non-negative  event  cost  if. 

Definition  6:  The  signed  real  measure  p  of  a  singleton 
string  set  {5}  c  L(c/,,  qj)  c  L(Gj)  e  2E*  is  defined  as 

MM)  =  if M  qdxiqj)  Vs  e  L{qh  qj). 


The  signed  real  measure  of  L{qh  qj}  is  defined  as 
li(L(qi,qjj)=  MM) 

seL(qt,  qj) 


and  the  signed  real  measure  of  a  DFSA  Gi,  initialized  at 
the  state  q,  e  Q,  is  denoted  as 

Pi  =  p(L(Gjj)  =  qj))- 

Definition  7:  The  state  transition  cost  of  the  DFSA  is 
defined  as  a  function  tv.  Q  x  Q  [0,  1)  such  that 

n(qj,  qk) 

0  if  {cr  G  E :  S(qj,  a)  =  qk}=0 

—  qj)  =  iCjk  otherwise 

a€’L-.S(q],a)=qk 

(2) 


It  has  been  shown  in  (Ray  2005  and  Ray  et  al.  2005) 
that  the  measure  pt  =  p{L{Gjj)  of  the  language  L{Gj), 
with  the  initial  state  qt,  can  be  expressed  as 
//;  --  y,^iiHi  -  //  where  Xi-x(qd ■  Equivalently,  in 
vector  notation 

/<  =  n/i  +  x=4/i  =  [ii-nr1x,  (3) 

where  the  measure  vector  p  =  [p\  p2  ■■■  p,,]1  and 
the  characteristic  vector  x  — [Xi  X2  •••  Xu]1'-  and  the 
condition  jjj  ify  <  1  e  /  (see  Definition  5)  is  sufficient 
for  the  inverse  to  exist. 

Although  the  preceding  analysis  reported 
in  (Ray  2005  and  Ray  et  al.  2005)  was  intended  for 
non-probabilistic  regular  languages,  the  event  costs  can 
be  easily  interpreted  as  occurrence  probabilities. 
As  such  the  n-matrix  is  analogous  to  the  morph 
matrix  of  a  Markov  chain  in  the  sense  that  an  element 
if y  represents  the  probability  of  the  /th  event  occurring 
at  the  ith  state  with  the  exception  that  the  strict  inequal¬ 
ity  condition  V.if,y  <  1  is  enforced  instead  of  satisfying 
the  equality.  Equivalently,  the  Il-matrix  is  analogous  to 
the  state  transition  probability  matrix  of  a  Markov 
chain  in  the  sense  that  an  element  njk  is  analogous  to 
the  transition  probability  from  state  qj  to  state  qk  with 
the  exception  that  the  strict  inequality  condition 
Kjk  <  1  is  enforced  instead  of  satisfying  the 
equality.  It  follows  that  the  preceding  analysis  is  applic¬ 
able  to  the  case  of  terminating  probabilistic  languages 
(Garg  et  al.  1992a,  b)  that  have  a  strictly  non-zero 
probability  of  termination  at  each  state. 

Let  E"  denote  the  set  of  all  unmodelled  events  in  the 
terminating  plant.  A  new  unmarked  absorbing  state 
q„+ 1,  called  the  dump  state  (Ramadge  and  Wonham 
1987),  is  created  and  the  transition  function  S  is  extended 
to  <5ext  :  (Q  U  {<?„+! })  x  (E  U  E")  ->  (Q  U  {qn+x}).  The 
residue  6j—  1  —  jjjk  njk  denotes  the  probability  of  transi¬ 
tion  from  qj  to  qn+\.  Consequently,  the  Il-matrix 
(see  Definition  7)  is  augmented  to  obtain  the  stochastic 
state  transition  probability  matrix  as 


7Tn 

7Tl2 

...  7T \n 

01- 

7T21 

7*22 

...  71 2« 

02 

(4) 

ktn\ 

7tn2 

•  •  •  Ttnn 

On 

_  0 

0 

0  ... 

1  . 

The  state  transition  cost  matrix,  denoted  as  Il-matrix,  is 
defined  as 


7Tn 

7t[n 

71  n\ 

Since  the  dump  state  qu+\  is  not  marked  (Ramadge  and 
Wonham  1987),  it  follows  from  Definition  4  that  the 
corresponding  state  weight  Xn+ 1  =  0.  Hence,  the 
/-vector  is  augmented  as 

Xaug  =  [XT  Of.  (5) 
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Denoting  ©  =  [0\  02  ■  ■  ■  0„]r,  where  0,-  e  (0, 1)  is  the 
probability  of  transition  from  the  state  cp  to  the  dump 
state,  it  follows  from  equations  (4)  and  (5)  that  the 
measure  of  the  augmented  system  (Chattopadhyay  and 
Ray  2006a)  is 

A*aug(®)  =  [M®)r  0]r  (6) 

Then,  the  event  cost  matrix  (see  Definition  5)  and  the 
state  transition  cost  matrix  (see  Definition  7)  can  be 
represented  as 


vector  i  becomes  one-dimensional  and  can  be  assigned 
as  x  =  1;  consequently,  the  measure  vector  p(Q) 
degenerates  to  the  scalar  measure  0_1.  To  alleviate  the 
singularity  of  the  matrix  operator  [I  — 11(0)]  at  0  =  0, 
the  measure  vector  fi(6)  in  (3)  is  normalized  with  respect 
to  0_1  to  obtain  the  renormalized  measure  vector. 

Definition  8:  The  renormalized  measure  is  defined  as 

v(0)  =  0M0)  =  0[D-n(0)r1x  (10) 

and  it  follows  from  (8)  that 


n(0)  = 


Diag[0] 


P  and  n(0)  = 


Diag[0] 


P, 

(7) 


0ft*ug(0)  =  [v(0)r  Of.  (11) 


where  P  and  P  are  both  stochastic  matrices  (Bapat  and 
Raghavan  1997),  i.e.,  EyPy  =  1  Vi  e  {1, . . .  ,m]  and 
£/Py  =  1  Vie  {l,...,n}. 

If  the  probability  of  termination  (or  equivalently  the 
probability  of  transition  to  the  dump  state)  is  equal 
for  all  states,  cp  e  Q ,  i.e.,  0,-  =  0  Vi  e  {1,2, . . .  ,n},  then 
equation  (6)  is  expressed  as 

0aug(?>  =  [mT  Of  (8) 

Consequently,  II  and  II  in  equation  (7)  are  represented  as 

n(0)  =  (l-0)P  and  n(0)  =  (l-0)P  (9) 

where  0  is  the  uniform  probability  of  termination  at  all 
states;  and  both  P  and  P  retain  the  properties  of  stochas¬ 
tic  matrices  (Bapat  and  Raghavan  1997). 

2.2  Renormalization  of  language  measure 

The  notion  of  language  measure  has  been  recently 
extended  to  non-terminating  models  by  assuming  a  uni¬ 
form  non-zero  probability  of  termination  (0)  at  each 
state,  renormalizing  the  language  measure  vector  with 
respect  to  the  probability  of  termination  and  computing 
the  limit  of  the  renormalized  measure  (Chattopadhyay 
and  Ray  2006a)  as  0  ->  0+.  As  the  probability  of  termi¬ 
nation  approaches  zero  (i.e.,  0  —*■  0+),  and  the  plant 
coincides  with  the  desired  non-terminating  model  in 
the  limit.  The  construction  of  renormalized  measure  is 
briefly  outlined  below. 

The  regular  language  generated  by  the  DFSA  under 
consideration  is  a  sublanguage  of  the  Kleene  closure 
E*  of  the  alphabet  E,  for  which  the  automaton  states 
can  be  merged  into  a  single  state.  In  that  case,  the 
state  transition  cost  matrix  11(0)  degenerates  to  the 
1  x  1  matrix  [1—0]  and  the  normalized  state  weight 


3.  Optimal  control  problem:  formulation 

The  following  notations  are  needed  for  elementwise 
comparison  of  finite-dimensional  vectors  and  matrices 
for  the  analysis  developed  in  the  sequel. 

Notation  1:  Let  V°  and  Vh  be  ( m  x  n)  real  matrices. 
The  following  elementwise  equality  and  inequalities 
imply  that 

(v°v  =  V*  )  Vi e  {1, . . . ,n}  Vy  e  {1, . .  .,m] 
(W E  vh\  O  vA  3/e{l,...,n},ye{l,...,m] 
(v^jA  O  in  >  \A  Vi  e  {1, . . .  ,n}  Y/e  {1, . . .  ,m] 

(r  >E  Vh\  O  ^  >  A  V/e{l,...,n}V/e{l,...,m}. 

For  the  terminating  plant,  investigated  in  (Ray  2005 
and  Ray  et  al.  2005),  the  optimal  supervisor  selectively 
disables  controllable  transitions  by  setting  their  occur¬ 
rence  probabilities  to  zero.  This  implies  that  if  II*  and 
n  are  the  transition  probability  matrices  for  the  opti¬ 
mally  supervised  plant  and  the  unsupervised  plant, 
respectively,  then 

n*gEn,  i.e.,  n*j  <  Tty. 

Since  for  any  non-trivial  supervisor,  there  is  at  least  one 
disabled  transition  in  the  supervised  plant,  i.e., 

3  i,j  such  that  i r,  ,■  >  0  and  n* ,  —  0 

■>J  l,j 

it  follows  that  if  the  unsupervised  plant  is  non¬ 
terminating,  then  any  non-trivial  supervision  will  result 
in  a  terminating  model.  The  policy  of  Kumar  and 
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Garg  2001  maintains  the  non- termination  property  by 
proportionately  increasing  the  probabilities  of  the 
remaining  enabled  events  at  the  state  at  which 
the  disabling  action  is  applied.  The  first  issue  here  is 
that  the  supervisor  must  be  able  to  affect  the  event 
occurrence  probabilities,  which  is  more  than  just  inhibit¬ 
ing  a  transition.  The  second  issue  is  that  there  is  a 
possibility  of  disabling  all  events  defined  at  a  given 
state  if  all  these  events  are  controllable.  In  that  case, 
the  row  sum  cannot  be  maintained  at  1  as  it  becomes 
strictly  equal  to  zero.  Thus,  it  is  necessary  to  impose 
special  constraints  on  the  unsupervised  plant  to  circum¬ 
vent  this  situation.  This  paper  investigates  an  alternative 
approach  as  described  below. 

Definition  9  (control  philosophy):  Disabling  any 
transition  a  at  a  given  state  q  results  in  reconfiguration 
of  the  automaton  structure  as:  Set  the  self-loop 
S(q ,  a)  =  q  with  the  occurrence  probability  of  a  from 
the  state  q  remaining  unchanged  in  the  supervised  and 
unsupervised  plants. 

This  is  equivalent  to  adding  a  self-loop  to  the  state  at 
which  the  event  is  being  disabled,  with  the  same  occur¬ 
rence  probability  as  the  disabled  transition. 

Preposition  1:  For  the  control  philosophy  in 
Definition  9,  a  supervised  plant  is  non-terminating  if 
and  only  if  the  unsupervised  plant  is  non-terminating. 

Proof:  The  proof  follows  from  two  lemmas. 

Lemma  1:  Each  row  sum  of  the  II-matrix  remains 
unchanged  after  supervisory  actions  for  the  control 
philosophy  in  Definition  9. 

Proof:  Let  II  and  IIT  be  the  transition  probability 
matrices  for  the  unsupervised  and  supervised  plants, 
respectively.  Let  there  be  exactly  one  disabled  transition, 
in  which  a  (controllable)  event  a  at  state  qt  is  disabled 
and  let  the  occurrence  probability  of  cr  at  state  qt  be  it. 
If  S(qi,  cr)  =  qk,  then  it  follows  that 


nf  =  n  + 


-o  o  ■■■ 

0  ■  ■  ■  if 
:  0 

0  . 


kth  column 

I 

■■  0  0" 

—it  0 
'.  0  : 

.  0 


<—  z'th  row 


implying  V  jrJ-  =  V-7Ty  Vf  The  proof  follows  by 
induction  on  the  number  of  disabled  events.  □ 


Lemma  2:  Self-loops  cannot  be  disabled. 

Proof:  Lor  the  control  philosophy  in  Definition  9, 
disabling  a  self-loop  at  any  given  state  causes  regenera¬ 
tion  of  the  self-loop  at  the  same  state  with  identical 
occurrence  probability.  □ 

It  is  evident  from  the  above  two  lemmas  that  each  row 
sum  of  the  reconfigured  II-matrix  remains  invariant. 
The  proof  of  Proposition  1  is  thus  complete.  □ 

Remark  1:  The  control  philosophy  in  Definition  9  is 
natural  in  the  following  sense.  If  <7,  „  qk,  and  the 

controllable  event  a  is  disabled  at  state  qh  then 
the  sole  effect  of  the  supervisory  action  is  to  prevent 
the  plant  from  making  a  transition  to  the  state  qk- 
That  is,  the  plant  is  forced  to  stay  at  the  original  state 
qt  and  this  is  represented  by  the  additional  self-loop  at 
state  qf  instead  of  the  original  arc  from  <7,-  to  qk. 

The  notion  of  controllability  is  now  clarified  in  the 
context  of  the  present  paper. 

Definition  10  (controllable  transitions):  Lor  a  given 
plant,  transitions  that  can  be  disabled  in  the  sense  of 
Definition  9  are  defined  to  be  controllable  transitions 
in  the  sequel. 

In  view  of  Definition  10,  controllability  becomes  state- 
based,  i.e.,  transitions  labelled  by  the  same  event  may  be 
controllable  from  one  state  and  uncontrollable  from 
some  other  state.  This  implies  that  the  event  alphabet 
E  cannot  be  partitioned  into  uncontrollable  and 
controllable  events  sets  as  proposed  in  Ramadge  and 
Wonham  (1987).  Thus,  a  controllable  transition  qt^qk 
refers  to  a  triple  {qt,  a,  qk}  and  the  set  of  all  such 
transitions  is  denoted  by  c€. 


3.1  Model  specification 

Plant  models  considered  in  this  paper  are  deterministic 
finite  state  automata  (DLSA)  with  well-defined  event 
occurrence  probabilities.  In  other  words,  the  occurrence 
of  events  is  probabilistic,  but  the  state  at  which  the  plant 
ends  up,  given  a  particular  event  has  occurred,  is 
deterministic.  Furthermore,  no  emphasis  is  laid  on  the 
initial  state  of  the  plant  and  it  is  assumed  that  the 
plant  may  start  from  any  state.  Furthermore,  having 
defined  the  characteristic  state  weight  vector  7,  it  may 
not  be  necessary  to  specify  the  set  of  marked  states, 
because  if  //  =  0,  then  <7,-  is  not  marked  and  if  x;  /  0, 
then  qt  is  marked.  Therefore,  plant  models  with  an 
arbitrary  uniform  termination  probability  6  e  (0, 1), 
i.e.,  6j  —  6  Vz  e  { 1,2,...,  n },  can  be  completely  specified 
by  a  sextuple  as 


G(0)  =  (e,E,a,  n(0),x,«?),  (12) 


2006  style  [23.4.2007-5:0 lpm]  [Revised  Proof]  [1-20]  [Page  No.  6]  {TANDF_REV}TCON/TCON_A_228536.3d 


(TCON)  TCONA  228536 


Optional  control  of  probabilistic  finite-state  systems 


7 


where  11(0), ^  is  the  occurrence  probability  of  event  a,  from 
state  q,  and  t  11(0), ^  =1—0  Vz.  An  application  of  (7) 
with  uniform  uniform  termination  probability  0  yields 
an  alternative  representation  of  the  sextuple  in  (12). 

G(9)=(Q,'E,8,(  l-0)P,Z,r),  (13) 

where  P  is  the  the  morph  matrix  of  the  underlying 
Markov  chain. 

As  0  — »  0+,  the  resulting  non-terminating  plant  model 
is  denoted  as 


G(0)  =  (Q,'Z,8,P,x,'C).  (14) 


Definition  11:  Given  0  e  (0, 1),  a  terminating  plant 
G(0)  —  (Q,  E,  <5,  (1  —  0)P,  x,  &)  is  defined  to  be  the 
0-neighbour  of  the  non-terminating  plant 

G(0)  =  (g,£,  5,  P,x,n 

For  a  given  non-terminating  plant  (7(0)  and  a  fixed 
0o  e  (0, 1),  there  is  exactly  one  0o-neighbour  G(0o). 

Notation  2:  Let  0  e  (0, 1)  be  the  unform  probability  of 
termination  for  a  terminating  plant  (7(0)  =  (Q,  E,  <5, 
(1  —  0)P,  x,  '&)■  Let  P  be  the  state  transition  probability 
matrix  of  the  underlying  Markov  chain,  which  is  gener¬ 
ated  from  8  and  n  (see  equation  (2)).  Then,  the  (renor¬ 
malized)  language  measure  vector  (see  Definition  8) 
is  obtained  as 


v(0)  =  0[o-(l  -0)P]  ‘x  (15) 

where  (1  —  0)P  is  the  sub-stochastic  transition  probabil¬ 
ity  matrix  for  the  terminating  plant.  Similarly,  for  a  non¬ 
terminating  plant  G(0)  =  (Q,  E,  5,  P,  x,  rC)  having  the 
stochastic  transition  probability  matrix  P,  the  (renorma¬ 
lized)  measure  vector  (Chattopadhyay  and  Ray  2006a)  is 
denoted  as 


v(0)  =  lim  v(0)  =  lim  0 

<9^0+  6—>0+ 


1-1 


(1  -  0)P 


(16) 


In  the  sequel,  renormalized  measure  v  in  equations  (10) 
and  (11)  is  referred  to  as  measure  for  brevity. 


Definition  12:  For  an  unsupervised  (non-terminating) 
plant  (7(0)  =  (Q,  E,  8,  P,  x,  C),  let  Gt  and  G+  be  the 
supervised  plants  with  sets  of  disabled  transitions, 
3ft  c  'z?  and  3^  c  (/f  respectively,  whose  measures  are 
v’1'  and  v+.  Then,  the  supervisor  that  disables  3^  is 
defined  to  be  superior  to  the  supervisor  that  disables 
if  v’1’  =EV-f  and  strictly  superior  if  v’1’  >ev*. 

Definition  13  (Optimal  supervision  problem):  Given  a 
(non-terminating)  plant  G(0)  =  (Q,  E,  8,  P,  x,  ’&),  the 
problem  is  to  compute  a  supervisor  that  disables  a 
subset  3*  c  c€ ,  such  that 

v*  vf  V#  c  % 

where  v*  and  F  are  the  measure  vectors  of  the  supervised 
plants  G*  and  G’1'  under  3*  and  3'' ,  respectively. 

Remark  2:  For  a  non-trivial  plant  G(0)  =  (Q,  E,  5, 
P,X,^)  (i.e.,  \Q\  >  1),  there  may  exist  two  supervisors 
that  are  not  comparable  in  the  sense  of  Definition  12. 
For  example,  given  a  two-state  unsupervised  plant  G, 
if  Gf  and  G*  are  supervised  plants  under  two  different 
supervisors  with  disabled  transition  sets,  3 ^  and  3^, 
respectively,  then  the  following  situation  may  occur  for 
the  indices  i  f  j. 

f  >  v<)  A  (v)  <  <i). 

f  t 

where  vj  and  vj  are  the  zth  elements  of  the  measure  vec¬ 
tors  for  G1'  and  G+,  respectively.  It  is  shown  in  the  next 
section  that,  for  a  given  plant,  an  optimal  supervisor 
(in  the  sense  of  Definition  13)  does  exist  for  which  the 
measure  vector  is  elementwise  greater  than  or  equal 
to  the  measure  vector  of  the  plant  under  any  other 
supervision  policy. 

Terminating  plant  models  have  sub-stochastic 
transition  probability  matrices  (see  Definition  7).  By 
postulating  the  existence  of  unmodelled  transitions, 
such  plants  can  be  transformed  to  non-terminating 
models  as  explained  below.  For  uniform  termination 
probability  0  e  (0, 1),  equations  (8)  and  (11)  suggest 
the  possibility  of  computing  optimal  supervision 
policies  for  terminating  plants  based  on  the  analysis  of 
non-terminating  plants. 


3.2  Construction  of  an  optimal  supervisor 

A  supervisor  disables  a  subset  of  the  set  of  controlla¬ 
ble  transitions  and  hence  there  is  a  bijection  between  the 
set  of  all  possible  supervision  policies  and  the  power  set 
2r .  That  is,  there  exists  2|r|  possible  supervisors  and 
each  supervisor  is  uniquely  identifiable  with  a  subset  of 
and  the  language  measure  v  allows  a  quantitative 
comparison  of  different  supervision  policies. 


4.  Optimal  control  problem:  solution 

This  section  presents  a  solution  to  the  optimal  supervi¬ 
sion  problem  by  assuming  a  uniform  non-zero  probabil¬ 
ity  of  termination,  0,  at  each  state.  Then,  it  is  shown  that 
the  solution  for  the  corresponding  non-terminating 
plant  can  be  obtained  from  the  control  policy  of  the 
terminating  plant  and  the  bounds  on  the  derivatives  of 
the  language  measure  (see  Appendix  A). 
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Let  0  e  (0, 1)  be  the  uniform  termination  probability  of 
an  unsupervised  plant  G(0)  —  ( Q ,  E,  8,  (1  —  0)P,  %,  ff). 
The  resulting  (substochastic)  state  transition  cost 
matrix  is  11(0)  =  (1  —  0)P.  For  such  plants  with  uniform 
non-zero  termination  probability,  the  following  lemma 
states  existence  of  an  augmented  plant  model. 

Lemma  3:  For  the  terminating  plant  G(Q)  — 
(Q,  E,  S,  (1  —  0)P,  x,  <&),  let  the  corresponding 
augmented  non-terminating  plant  be  Gaug  =  (Qa ug, 
L;iug,  ^aug-*  naug,Xaug> T)-  Let  v*(6)  and  vf(0)  be  the  mea¬ 
sures  of  the  terminating  plant  with  the  respective  sets  of 
disabled  transitions  3*  c  c€  and  3ft  c  c€.  Then, 

3S)*  c  s.t.  v*(0)gEvt(0)  V#  c  f  V0  e  (0, 1) 

(17) 


Proof:  It  follows  from  equation  (15)  in  Notation  2  that 

vf  —  V  =  0[D  -  nf]_1— (9[D  -  m_13c 

=  e[D  -  rfi]_1([D  -  m  -  [n  -  nf])[D  -  nr  i* 

=  0[n-nt]_1(nt -n)v. 

Defining  the  matrix  A  =  iT  —  II,  and  the  /th  row  of  A  as 
A,-,  it  follows  that 

A,-7  v  =  A ijvj  -  ePurv  ( 1 8 ) 

j  j 

where 


which  implies  that  an  optimal  supervisor  for  Gaug  exists 
(in  the  sense  of  Definition  13)  which  disables  3*  c  ^ . 


r  ]  = 


(Vi  -  vj)  if  Vi  >  vj 
0  if  Vj  =  Vj  Ty  ’  0  Vi,  /. 


Proof:  The  first  n  elements  of  the  measure  vectors  of 
the  augmented  plant  and  the  unaugmented  plant  are 
identically  equal  as  seen  in  equation  (11).  Then,  the 
proof  follows  from  Definition  12.  □ 

The  remainder  of  this  section  derives  an  algorithm  for  a 
supervision  policy  that  elementwise  maximizes 
the  measure  of  the  terminating  plant  G(G).  Lemma  3 
guarantees  that  the  optimal  policy  is  based  on  a  non¬ 
terminating  plant. 

Proposition  2  (Monotonicity):  Let  11(0)  and  v(0)  be  the 
state  transition  cost  matrix  and  the  measure  vector  of 
an  unsupervised  plant  G(0)  =  (Q,  E,  8,  (1  —  0)P,  7, 
respectively.  Let  a  supervisor  be  constructed  to  reconfi¬ 
gure  the  plant  by  disabling  a  set  of  controllable  transi¬ 
tions  3?  c  ^  such  that  IT  is  modified  to  IT  by 
following  Algorithm  1.  Then,  denoting  the  measure 
vector  of  the  supervised  plant  by  T,  it  follows  that 
=£e  v;  and  equality  holds  if  and  only  if  iT  =  II. 


Algorithm  1:  Monotonic  Increase  of  Measure  v 


begin 

for  i  =  l  to  n  do 


for  j  =  1  to  n  do 

nf  =  n„  +  fa 
nf  =  n  u-Pij 
n7=n„ 
nt  =  n„ 

n0'  =  ny-fr; 

n,;  =  11;,  +  Pij 

endfor 

endfor 


end 


if  Vj  >  Vj  with  fijj  >  0 


if  Vj  =  Vj 


if  Vj  <  Vj  with  j 3jj  >  0 


(Vj  -  Vj)  if  Vj  <  Vj 


Since  IT/  =  11^-;  V/,  k,  it  follows  from  non¬ 

negativity  of  II,  that  [/—  iTp^EO.  Since  fij  >  0  Vi,  it 
follows  that  A,7v  >  0  V/  ^v^ev.  For  Vj  A  0  and 
A  as  defined  above,  Afvk  —  0  if  and  only  if  A  =  0. 
Then,  iT  =  II  and  vT  =  v.  □ 

Corollary  1:  Under  an  identical  situation  to  that 
assumed  in  the  statement  of  Proposition  2,  let  the 
plant  be  reconfigured  as  given  in  Algorithm  2.  Then, 
denoting  the  measure  vector  of  the  modified  plant  by 
vT  it  follows  that  v^Se  v;  and  equality  holds  if  and 
only  if  IT  =  II. 


Algorithm  2:  Monotonic  Decrease  of  Measure  v 


begin 

for  i  =  1  to  n  do 
for  j=l  to  n  do 


endfor 
endfor 


end 


nf-FIy+ft; 
nt  =  n„  -fa 
n?/  =  n,7 

nt  =  n„ 


Hry  =  n.;-ft/ 

nt  =  if,  +  fij. 


if  Vj  <  Vj  with  fijj  >  0 


if  Vj  =  Vj 

if  Vj  >  Vj  with  fijj  >  0 


Proof:  The  proof  is  similar  to  that  of 
Proposition  26.  □ 

Proposition  2  facilitates  formulation  of  the  algorithm 
for  computing  an  optimal  supervisor  for  plants  with 


2006  style  [23.4.2007-5:01pm]  [Revised  Proof]  [1-20]  [Page  No.  8]  [TANDF_REV}TCON/TCON_A_228536.3d  (TCON)  TC0N_A_228536 


Optional  control  of  probabilistic  finite-state  systems 


9 


uniform  non-zero  probability  of  termination  at 
each  state.  Let  the  Arth  iteration  of  the  algorithm  com¬ 
pute  a  set  c  ^  of  controllable  transitions  to  be 
disabled  in  the  sense  of  the  control  philosophy  in 
Definition  9.  The  language  measure  vector  computed 
in  the  A’th  iteration  of  the  algorithm  is  denoted  by  riA4 
The  algorithm  terminates  at  the  ( k  +  l)th  iteration  if 
—  q )[*+i]  ^  which  is  the  optimal  set  of  disabled  transi¬ 
tions  computed  by  the  algorithm  and  is  denoted  by  &*. 
The  algorithm  is  started  with  the  unsupervised  plant 
(i.e.,  with  all  controllable  transitions  enabled)  and 
hence  ^°1  —  0.  A  formal  description  is  given  in 
Algorithm  3. 


Algorithm  3:  Optimal  Supervisor  Algorithm 
input  :  n(S)  =  (1  -  0) P,  x, 
output:  Optimal  set  of  disabled  transitions 

1  begin 

2  Set  =  0  ;  /*  Initial  disabling  set  */ 

3  Set  k  =  1  ; 

4  Set  Terminate  =  false  : 

5  while  (Terminate  ==  false)  do 

6  Compute  vM  ; 

7  for  j  =  1  to  n  do 

8  for  k  =  1  to  n  do 

9  Disable  all  controllable  transitions  q^  A  q j 
such  that  Vj  <  v ^  ; 

10  Enable  all  controllable  transitions  q^  A  it 

such  that  Vj  S  vj  ; 

endfor 
endfor 

Compute  ;  /*  kth  disabling  set  */ 

if  ==  then 

Terminate  =  true  ; 
else 


11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21  end 


I  k  = 

endif 
endw 

rtf*  _  gilt]  . 


k  +  1 


/*  Optimal  disabling  set  */ 


Proposition  3:  Let  v[A]  be  the  language  measure 
vector  computed  in  the  A’th  iteration  of  Algorithm  3. 
The  measure  vectors  computed  by  the  algorithm 
form  an  elementwise  non-decreasing  sequence,  i.e., 
vh+i]  sE  vm  yfc. 

Proof:  Let  the  state  transition  probability  matrix  in 
the  A'th  iteration  of  Algorithm  3  be  denoted  by  II[A4 
Then,  the  matrix  I#+1]  is  generated  from  1 1 A  by  follow¬ 
ing  the  procedure  as  described  in  Proposition  2.  Hence, 
vh+ngj-vW.  □ 

Proposition  4  (effectiveness):  Algorithm  3  is  an 
effective  procedure  (Hopcroft  et  al.  2001),  i.e.,  it  is 
guaranteed  to  terminate. 

Proof:  Let  G{6)  —  ( Q ,  £,  S,  (1  —  0)P,  /,  rC)  be  the  unsu¬ 
pervised  plant  and  let  Card( (€)  =  Denoting  the 


set  of  all  permutations  of  the  vector  [12  l]T  by 
#(£),  a  function  2T — is  defined  as 


1.  V#  c  9g,  (vt  >  vj,  >  •  •  •  >  vj:  ) 

=>{&&)  =  [il  ri  ■  ■  ■  inf) 

2-  (vl  =  4,)  A  (4  >  h) 

=>(if  —  4  and  §(#),  =  it  then  s  >  t). 

Let  be  the  measure  vector  computed  in  the  A’th  itera¬ 
tion  of  Algorithm  3.  Then,  v[A]  =  v[A+1]  implies  that 
Algorithm  3  terminates  in  A  +  1  iterations  according  to 
its  stopping  rule. 

Next  let  and  be  the  disabling  sets  in  itera¬ 
tions  A’i  and  k2,  respectively.  If  £(5^A|])  =  f(£?>^2]),  then 
V[fci+i]  =  v[A'2+l].  Since  £(®[Ai])  =  §(S>^),  it  follows  from 
the  definition  of  §  that  if  v\k  1  *  >  \}k'\  then  i//c^  J-k^ . 
If  v[A^  >  then  controllable  transitions  c]i-+aqj  are 
disabled  in  both  iterations  k\  +  1  and  A^+l.  If 
v|A|]  =  vjAl],  then  disabling  or  enabling  controllable 
transitions  qi^-aqj  does  not  affect  the  measure  vector. 
Hence,  it  follows  that  v[A'1+1]  and  v[A'2+1]  can  be  obtained 
by  disabling  the  same  set  of  controllable  transitions, 
thus  implying  riA|+1]  =  riA2+14  Since  the  measure 
vectors  can  repeat  only  at  the  final  iteration, 
Algorithm  3  is  guaranteed  to  terminate  within 
Cardiff (fi))  —  l\  iterations.  Therefore,  effectiveness  of 
Algorithm  3  is  established.  □ 

Next  it  is  established  that  Algorithm  3  is  correct  in  the 
sense  that  an  optimal  supervision  policy  is  generated. 

Proposition  5  (Optimality):  For  a  terminating  plant 
G(6)  —  ( Q ,  E,  S,  (1  -  0)P,  x,  rri),  the  supervision  policy 
computed  by  Algorithm  3  is  optimal  in  the  sense  of 
Definition  13. 

Proof:  Let  G(6)  have  the  state  transition  cost  matrix  n, 
measure  ri°J,  no  disabled  events,  i.e.,  @°  =  0.  Let  G(8)  be 
configured  as  the  supervised  plant  G*(9 )  by  application 
of  Algorithm  3  when  it  stops. 

Let  G'  be  another  configured  plant  distinct  from  G*. 
Let  2>*  c  and  (f'  c  rC  be  the  respective  sets  of 
disabled  transitions  and  v*  and  be  the  respective 
measures  for  G*  and  G' ;  and 

Let  the  following  set  differences  be  denoted  as: 
A  and  V®  =  ®^\®*.  An  application  of 

Algorithm  3  yields 

•  V/,  /  r*  >  v*  =>■  all  controllable  transitions  cp  f  qj  are 
disabled. 

•  V/,  /  v*^v*  =>■  all  controllable  transitions  qifqj  are 
enabled. 
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To  change  the  plant  configuration  from  G *  to  G\  all 
transitions  in  A3  are  enabled  and  all  transitions  in 
V3  are  disabled.  Since  any  such  change  requires  us  to 
either  disable  a  transition  q,  — >  „  qj  where  v *  <  v*  or 
enable  a  disabled  transition  qi-*aqj  where  v *  >  v*,  it 
follows  from  Corollary  1  that  v’1'  =e  v*. 

Since  G'  is  an  arbitrary  configuration  distinct  from 
G*,  it  follows  that  G *  is  an  optimal  supervision  policy 
in  the  sense  of  Definition  13.  □ 

In  the  reported  work  on  discrete  event  control  of  non- 
probabilistic  regular  languages  (e.g.,  (Ramadge  and 
Wonham  1987)),  the  emphasis  is  on  computing  the 
maximally  permissive  supervisor  in  the  sense  that  the 
supervised  plant  language  is  the  supremal  controllable 
sub-language  of  the  specification.  A  similar  approach 
is  taken  for  probabilistic  regular  languages  (Garg 
1992a,  b).  In  contrast,  the  measure-theoretic  concept 
in  this  paper  computes  a  policy  that  maximizes  the 
elements  of  the  language  measure  vector  elementwise 
to  find  a  supervisor  with  maximal  performance. 
Proposition  5  shows  that  there  exists  at  least  one  optimal 
supervisor.  Now  it  is  shown  that  the  optimal 
supervisor  computed  by  Algorithm  3  is  unique  in  the 
sense  of  being  maximally  permissive  among  all 
policies  that  guarantee  optimal  performance  of  the 
supervised  plant. 

Proposition  6  (uniqueness):  Given  an  unsupervised 
plant  G(0),  the  optimal  supervisor  G*(0),  computed  by 
Algorithm  3,  is  unique  in  the  sense  that  it  is  maximally 
permissive  among  all  possible  supervision  policies  with 
optimal  performance.  That  is,  if  3*  and  3r  are  the  dis¬ 
abled  transition  sets,  and  v*  and  v'"  are  the  language 
measure  vectors  for  G*  and  an  arbitrarily  supervised 
plant  G’1’,  respectively,  then 

V*  =E  v+  c  #  C  c€.  (19) 

Proof:  If  G*  and  G'  are  distinct,  then  3#  ^  3*.  Given 
v*  =E  v#,  iet  G*  be  reconfigured  to  G1’  by  disabling  and/or 
re-enabling  appropriate  controllable  transitions.  It 
follows  from  equation  (18)  that 

o  =  vf  —  v*  =  [a  -  n+]_1  (nt  -  n*)v* 

=►  (nt  -  II*)  v*  =  0.  (20) 

The  /th  element  of  (IIT  —  n*)v*  is  expressed  as  the 
finite  sum  of  real  numbers 


0=  (nt-n*y  =£rr, 


r=  1 


(21) 


where  0  <  r  <  2CW(Vf)  and  each  Tr  is  of  the  form: 


T.  = 


a(.(v*  —  v*)  >  0,  if  T\  arises  due  to  disabling 
qi~l  qj  for  some  qf  e  Q 

a'r(v*  —  v*)  >  0,  if  T'  arises  due  to  enabling 
qj  for  some  qj  e  Q 

(22) 


because  each  a'r  represents  event  occurrence 
probabilities  and  hence  are  positive,  and  the  the  logic 
of  disabling  and  re-enabling  follows  Algorithm  3. 
Therefore,  it  follows  from  equation  (22) 
that  T‘  =  0  Vr  e  {l,...,r}. 

Hence,  it  is  necessary  to  re-enable  controllable  transi¬ 
tions  qi-^-aqj  and  disable  the  self  loop  at  qi  such  that 
v]  —  Vj  for  reconfiguration  from  Gg  to  G*e.  Note  that 
all  such  transitions  are  guaranteed  to  be  enabled  in  G*e 
(see  line  10  in  Algorithm  3).  Therefore,  given  v*=ev1’, 
it  follows  that  3*  c  3^.  That  is,  G*(d)  is  unique  for 
all  6  e  (0, 1)  in  the  sense  that  the  configured  plant  is 
maximally  permissive  among  all  other  configurations 
that  yield  the  same  optimal  measure  v*(0).  □ 


4.1  Optimal  control  of  non-terminating  plants 

This  section  presents  the  optimal  supervision  problem 
for  non-terminating  plants  (i.e.,  with  termination  prob¬ 
ability  0=0  at  each  state)  having  the  structure 
G(0)  =  ( Q ,  £,  8,  P,  i,  rG)  and  the  corresponding  stochas¬ 
tic  transition  probability  matrix  is  P.  The  rationale 
for  working  on  a  terminating  plant,  instead  of  the 
non-terminating  plant  is  explained  below. 

By  maximizing  the  measure  v(0)  for  a  given  0  e  (0, 1), 
an  optimal  control  law  can  be  derived  based  on  the  state 
transition  cost  matrix  11(0)  =  (1  —  0)P  of  the  supervised 
plant  language  and  the  originally  assigned  /-vector. 
Such  an  optimal  control  law  is  sought  to  be  0-indepen- 
dent  in  the  sense  of  having  the  same  disabling  set 
3  c  for  a  given  range  of  0,  where  0  might  be  restricted 
to  be  not  too  far  away  from  0+.  On  the  other  hand,  from 
the  perspective  of  numerical  stability  and  accuracy  in 
computation  of  v(0)  (see  Definition  8),  it  is  desirable  to 
have  a  relatively  large  positive  value  of  0.  The  results 
derived  in  this  section  serve  toward  establishing  upper 
bounds  on  0  for  which  the  optimal  control  law  should 
be  0-independent  and  the  associated  computation  is 
numerically  stable.  The  main  objective  is  summarized 
below. 

A  uniform  non-zero  probability  of  termination  ft.  e  (0, 1)  is  to 

be  computed  such  that  the  terminating  plant  G(ft„)  and  the 
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non-terminating  plant  G(0)  shall  have  the  same  the  disabling  set 
3  c  t'.  However,  in  general,  their  measures  could  be  different, 
i.e.,  v(6».)  #E  v(0). 

Proposition  7:  Let  ( 1  —  0)P  and  v(0)  be  the  state  transi¬ 
tion  cost  matrix  and  the  measure  of  the  plant 
G(0)  =  (Q,  £,<5,(1  —  0)P,  3C,  n  Then,  for  all  qu  qj  e  Q, 
there  exists  0*.  e  (0, 1]  such  that  V0  e  (0, 0*),  the  sign 
of  (v,(0)  —  v/0))  is  fixed  (i.e.,  positive,  negative  or 
zero);  and  0t.  can  be  computed  as  an  explicit  function 
of  the  stochastic  matrix  P  and  state  characteristic 
vector  x- 

Proof:  Let  y,y(0)  =  v,-(0)  —  v;(0)  V0  e  (0, 1),  which  is  a 
smooth  function  of  0,  and  yf  0)  =  liny^0+y<y(0)-  The 
proof  is  based  on  the  following  two  cases. 

Case  1:  No  sign  change  of  yy(9)  in  (0, 1)  0*.  =  1. 

This  includes:  y,y(0)  —  0  and  (dkyij(9)/d9k)\g=0  =  0  for  all 
Jfc>0  because  y,y(0)  —  0  V0  e  (0, 1)  by  Proposition  A. 3. 

Case  2:  yy(0)  changes  sign  in  (0, 1);  and 

(dryij(6)/d9r)\e=0  =  Ay  f  0  for  some  integer  r>  0. 

If  r  =  0,  there  exists  ti  e  (0, 1)  such  that  y,j(ri)  —  0  for 
the  first  time.  If  r  >  0,  it  is  possible  that  yfO)  —  0. 
Then,  as  0  is  increased  from  zero,  y,y(0)  becomes  non¬ 
zero  and  there  exists  t\  e  (0, 1)  such  that  y,y(ri )  =  0 
again.  Smoothness  of  y,y(0)  necessitates  that 
(dryij(9)/d0r)\o=a*  —  0  for  some  0*  e  (0,  ti).  Then,  it 
follows  from  the  Mean  value  Theorem  that  there  exists 
r2  e  (0, 0*)  such  that 

9r+V#)i 

30'  +*  1 2  \e=T2  Q* 

for  the  given  r  >  0,  Proposition  A. 2,  triangular  inequal¬ 
ity,  and  the  relation  y,y(0)  =  v,(0)  —  vy(0)  yield 


(3^,(0)79001^0  -  ( dr vf9) / 30r+ 1 ) 1 0=o 

(r  +  1)!  2'+3 4 

info#  o 

[D  -P  +  a.f]  1 

ooy 

r+l 

|{[]  _  P  +  //]-■  x  -  ,/x},.  -  ([[- p  +  .y>]-‘x- 

sx^/^nii-p+^r'iu 


{[]— p  p — 

-{[o-p+<f|-I[]-[o-p+#r1]rz}y 


if  r  >  0. 

(23) 

□ 


Remark  3:  For  a  non-terminating  plant  G(0)  = 
(Q,  E,5,P,  x),  let  0 *  =  min/j  0*-.  Then,  the  plant  config¬ 
uration  obtained  by  applying  a  single  iteration  of 
Algorithm  3  to  the  ^-parameterized  plant  G(0)  = 
( Q ,  E,  <5,  (1  —  6) P,  x,  c€)  is  identical  for  all  0  <6*. 

The  procedure  of  computing  0*  is  summarized  as 
Algorithm  4. 

Proposition  8:  Complexity  of  computing  a  positive 
bound  for  6 *  is  0(n 3)  where  n  is  the  number  of 
plant  states. 


Algorithm  4:  Computation  of  the  Bound  0* 

input  :  P,  x 
output:  (9* 

1  begin 

2  Set  dir  =  l; 

3  Set  dcurr  =  0; 

4  Compute  & 


7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 


/*  See  Algorithm  6  in  Appendix  A 


I-P+; 


I  -  I  -  P  +  c 


*/ 

Compute  Mo  = 

Compute  Mi  = 

Compute  M2  =  inf  a±Q  ||[l  -  P  +  ||  ; 

for  j  =  1  to  n  do 
for  i-  1  to  n  do 

if  {&x)i  ~  (&X)j  ^  0  then 
|  9am  =  g MJ  |  (^X)i  -  (&X)j  | 

else 

for  r  =  0  to  11  do 

if  (M0x)i  *  ( M0x)j  then 
Break: 
else 

if  {MqM\x).  *  (MaM\x) .  then 
Break: 
endif 
endif 
endfor 
if  r  ==  0  then 

1  (M0- &>)x  ~  (M0- 3*)x  j!  . 


9ci 


else 


if 


n  ~  8 M2 

r  >  0  AND  r  <n  then 


dcurr  — 


Jcurr  ■ 

else 

|  dcurr  —  1  : 

endif 
endif 
endif 

0*  =  min(0*,  9curr)  ; 

endfor 
endfor 


2r+3M2 


end 
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Proof:  Referring  to  Algorithm  4,  the  part  within  the 
nested  For  loops  (lines  10  to  32)  is  executed  at  most  n2 
times  and  each  iteration  involves  only  single-iteration 
scalar  operations.  Thus  the  computational  complexity 
of  this  part  is  of  the  order  of  0(n2).  Lines  5  and  6  involve 
inversion  of  n  x  n  dimensional  non-singular  matrices 
and  hence  the  complexity  of  execution  is  of  the  order 
of  0(n 3).  Proposition  G  (see  Appendix)  guarantees 
that  the  complexity  of  computing  ZP  is,  in  general, 
of  the  order  of  0(n 3).  Line  7,  which  computes 
Mi  —  infa^o  || [D  —  P  -F  cko-^]-1  lloo>  is  a  search  problem. 
However,  since  M2  appears  only  in  the  denominator  of 
the  expressions  for  0curr,  it  follows  that,  if  for  some 
a  —  a0  7^  0  and  by  using 


M2 


[I  -  P  +  «(,,/] 


OO 


(24) 


it  is  possible  to  obtain  a  positive  lower  bound  of  0* 
in  Algorithm  4.  Since  the  computation  of 
|| [D  —  P  +  o'o-^]_1||00  is  of  the  order  of  0(n3)  due  to  the 
matrix  inversion,  it  is  concluded  that  a  positive  lower 
bound  of  0*  can  be  computed  with  a  complexity 
of  0(n3).  □ 

Remark  4:  It  is  shown  in  (Chattopadhyay  and  Ray 
2006a)  that  for  any  stochastic  matrix  P 


[0  — P  +  a.f\  1 


=  [D-P  +  .J/]  1 


S>  Va/0 


V-  (25) 

Using  M2  —  ||  [D  —  P  +  -^“'lloo  instead  of 
Mi  —  inf^o  II  [D  —  P  -F  (i.e.,  using  a  —  1)  in 

Algorithm  4  yields  a  value  which  satisfies  the  require¬ 
ment  stated  in  Remark  3  and  therefore  qualifies  as  0*. 
Thus,  the  major  advantage  of  this  approximation  is 
having  significantly  smaller  computational  complexity 
because  the  search  involved  in  computing  the  infimum 
is  avoided  at  the  cost  of  using  a  smaller  value  of  0*, 
which  may  make  subsequent  computation  of  measure 
slightly  more  difficult  due  to  possible  ill-conditioning 
(see  Definition  8). 


>([D-P  +  ^]  '-&>}  +  ( 


On  account  of  Proposition  7  and  Remark  3,  Algorithm  3 
is  modified  to  solve  the  optimal  supervision  problem 
for  non-terminating  plants  and  the  modified  version  is 
formally  presented  in  Algorithm  5. 


Algorithm  5:  Computation  of  Optimal  Supervi¬ 
sor 


input  :  P,  x, 

output:  Optimal  set  of  disabled  transitions  S’* 

1  begin 

2  Set  =  0  ;  /*  Initial  disabling  set 

3  Set  nl°]  =  ft  ;  /  *  Initial  event  prob.  matrix 

4  Set  0™  =  0.99; 

5  Set  k  =  1 

6  Set  Terminate  =  false  : 

7  while  (Terminate  ==  falsej  do 

8 


Compute  O' 


[*]. 


/*  Algorithm  4 


9 

10 

11 

12 

13 

14 


Set  nW  =  -2-^n^-1]; 

1-el)-11 

Compute  v'k‘  ; 

for  /  =  1  to  n  do 
for  /  =  1  to  n  do 

Disable  all  controllable  transitions  q,  —>  qj 

such  that  v[k]  <  v[k]  ; 

7  1 

Enable  all  controllable  transitions  c\i  — > 

such  that  iP  >  v[k]  ; 

7  “  1 


15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25  end 


endfor 

endfor 

Collect  all  disabled  transitions  in  ; 
if  Q)\k\  --  then 

|  Terminate  =  true  ; 
else 

|  k  =  k  +  1  ; 

endif 

endw 

;  /*  Optimal  disabling  set 


*/ 

*/ 


*/ 


*/ 


Proposition  9  (effectiveness):  Algorithm  5  is  an  effec¬ 
tive  procedure  (Hopcroft  et  al.  2001),  i.e.,  it  is  guaran¬ 
teed  to  terminate. 

Proof:  Comparison  of  Algorithm  3  and  Algorithm  5 
reveals  that  while  the  former  assumes  a  fixed  probability 
of  termination  0  at  each  state,  the  latter  modifies 
this  parameter,  denoted  as  9&\  at  each  iteration  k.  Let 
dmin  =min(0d],0p])  and  let  @[1](0mra)  and  ®[2](0min)  be 
sets  of  disabled  transition  at  the  first  and  second  itera¬ 
tions,  respectively,  for  the  terminating  plant  G(0min). 
Similarly,  for  the  non-terminating  plant  G-(0),  let 
S>bl(0)  and  2^(0)  be  the  sets  of  disabled  transitions 
at  the  first  and  second  iterations,  respectively.  It 
follows  from  Remark  3  that  2>bl(0)  —  7[1](0min)  and 
®[2](0)  =  2>[2](0min). 

Extending  the  above  argument  by  induction  based 
on  k  iterations  of  Algorithm  5  and  denoting 
0min  =  min(0W,...,0?1),  an  application  of  Algorithm  3 
on  a  terminating  plant  G(0min)  yields 

®w(o)  =  ®M(0m.n)  vre 
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Proposition  4  states  that,  for  an  arbitrary  plant, 
Algorithm  3  is  guaranteed  to  terminate  within  finitely 
many  iterations.  Hence,  Algorithm  5  is  an  effective 
procedure.  □ 

Next,  it  is  shown  that  the  plant  configuration  obtained 
by  Algorithm  5  is  optimal  in  the  sense  of  Definition  13. 

Proposition  10  (optimality):  For  a  non-terminating 
plant  G(0)  =  (Q,  £,  S,  P,  x,  *&)),  the  supervision  policy 
computed  by  Algorithm  5  is  optimal  in  the  sense  of 
Definition  13. 

Proof:  Let  the  set  of  disabled  transitions  computed  at 
the  /cth  iteration  Algorithm  5  be  denoted  by  3^  and 
the  termination  probability  be  denoted  by 
Let  the  set  of  disabled  transitions  at  the  convergence 
of  Algorithm  5  be  Let  0min  =  minre{i....j£) 

(0W,...,0M)>O. 

Let  G(0m  in)  be  a  terminating  plant  with 
II(0min)  =  (1  —  0min)P.  It  follows  from  the  proof  of 
Proposition  9  that  applications  of  Algorithm  3  to 
G(0m in)  and  Algorithm  5  to  G( 0)  yield  the  same  set  2> 
of  disabled  controllable  events  although  the  optimal 
measures,  being  0-dependent  would  be  different,  i.e., 
v(0min)  v(0). 

Proposition  5  implies  that  the  optimal  disabling  set 
for  a  plant  G(0)  generates  the  the  same  set  of  disabled 
controllable  transitions  for  all  0  <  6  =  0m in.  Because  of 
continuity  of  v(0)  with  respect  to  0,  it  is  argued  that 
G*(0)  is  optimal  in  the  sense  of  Definition  13,  i.e., 
v*  =§e  v1',  where  G^O)  is  obtained  by  arbitrarily  disabling 
controllable  transitions  in  G.  This  completes  the 
proof.  □ 

Next  it  is  shown  that  the  supervision  policy  computed  by 
Algorithm  5  is  unique  in  the  same  sense  as 
Proposition  6. 

Proposition  11  (Uniqueness):  Let  G(0)  be  an  unsuper¬ 
vised  non-terminating  plant  and  G*(0)  be  the  supervised 
plant  configured  by  Algorithm  5.  Then,  G*  is  unique  in 
the  sense  that  it  is  maximally  permissive  among  super¬ 
vised  plants  that  yield  optimal  performance  based  on 
0- neighbours  G(0)  of  G(0)  (see  Definition  11)  for  all 
0  e  (0, 0*),  where  0*  is  computed  by  Algorithm  4. 
Equivalently,  if  G’l’(0)  is  an  arbitrarily  supervised  plant, 
then  the  following  condition  holds: 

(v*(0)  vf(0))  /\  {{&*  c  3ft)  \/  (v*(0)#E  vf(0))), 

where  v  and  3>  denote  respective  language  measures  and 
sets  of  disabled  transitions. 


Proof:  It  follows  from  Proposition  10  that 
v*(0)  §e  vT(0).  It  also  follows  from  Proposition  that 
v*(0)§Evt(0)  for  0  €  (0,0*).  If  v*(0)  =e  vt(0),  then  G*(0) 
and  G'(ff)  are  both  optimal  supervised  configurations 
of  the  unsupervised  terminating  plant  G(0).  It  follows 
from  Proposition  6  that  S>*  c  git.  otherwise 
v*(0)^e  vf(0).  □ 

Proposition  12:  Computational  complexity  of 
Algorithm  5  is  of  the  same  order  as  that  of  Algorithm  3. 

Proof:  Algorithm  5  computes  0,  in  each  iteration  and 
complexity  of  this  computation  is  0(n 3),  where  n  is  the 
number  of  states  in  the  plant  (see  Proposition  8).  Each 
iteration  of  both  Algorithm  3  and  Algorithm  5  involves 
computation  of  the  measure  vector  v,  whose  complexity 
is  also  0(tf)  because  of  n  x  n  matrix  inversion.  Hence, 
computational  complexity  of  each  iteration  is  O(if) 
for  both  Algorithm  3  and  Algorithm  5.  Finally,  the 
argument  presented  in  Proposition  9  implies  that 
the  number  of  iterations  in  Algorithm  5  is  of  the 
same  order  as  that  in  Algorithm  3.  This  completes 
the  proof.  □ 

4.2  Testing  of  computational  complexity 

Proposition  4  shows  that  Algorithm  3  is  an  effective 
procedure  (Hopcroft  et  al.  2001),  i.e.,  the  solution  is 
guaranteed  to  converge  in  a  finite  number  of  iterations. 
Extensive  simulation  suggests  that  the  the  maximum 
number  of  iterations  for  Algorithm  3  is  actually  of  poly¬ 
nomial  order  in  n,  where  n  is  the  number  of  states  in  the 
unsupervised  plant.  The  result  is  illustrated  in  figure  6, 
where  the  maximum  number  of  required  iterations 
/max  is  plotted  against  number,  n,  of  plant  states.  For 
each  n,  10,000  simulation  runs  were  conducted  for 
synthesis  of  optimal  plant  configuration  with  randomly 
generated  entries  in  the  pair  ((1  —  0)P,  7);  and  /max 
was  chosen  to  be  the  maximum  number  of  iterations 
required  by  Algorithm  3  to  converge;  this  is  the  most 
conservative  case.  The  plot  in  figure  1  shows  a  distinct 
sub-linear  variation.  The  following  conjecture  is  made 
based  on  these  observations. 

Conjecture  1  (polynomial  convergence):  Given  a  termi¬ 
nating  plant  G(0)  with  a  uniform  non-zero  probability  of 
termination  0  at  each  of  the  n  plant  states, 

1 .  Algorithm  3  converges  in  at  most  0(n)  iterations. 

2.  Computational  complexity  of  Algorithm  3  is  bounded 
by  0(n4). 

Statement  2  in  Conjecture  1  follows  from  Statement  1 
and  the  following  facts:  Each  iteration  has  complexity 
of  0(n 3)  due  to  matrix  inversion  in  the  computation  of 


2006  style  [23.4.2007-5:01pm]  [Revised  Proof]  [1-20]  [Page  No.  13]  {TANDF_REV}TCON/TCON_A_228536.3d  (TCON)  TCON_A_228536 


14 


I.  Chattopadhyay  and  A.  Ray 


the  language  measure  vector,  and  matrix  inversion  has 
complexity  of  OUr')).  Combination  of  Conjecture  1 
and  Proposition  12  implies  that  Algorithm  5  converges 
in  0{n)  iterations  and  that  complexity  of  the  algorithm 
is  0(n4).  Similar  to  the  procedure,  described  above  for 
Algorithm  3,  10,000  random  simulation  runs  for  each 
n  were  conducted  for  testing  Algorithm  5.  Figure  2 
shows  the  plot  of  average  number  of  iterations  required 
to  converge  at  each  value  of  n  in  contrast  to  figure  1, 
where  the  maximum  number  of  iterations  is  potted.  As 
expected,  the  plot  of  figure  2  is  also  sub-linear. 

5.  Optimal  control  of  three  processor  message  decoding 

This  section  presents  the  design  of  a  discrete-event 
(controllable)  supervisor  for  a  multiprocessor  message 


Figure  3.  Arrangement  of  the  processor  links. 


decoding  system,  described  in  an  earlier 
publication  (Ray  et  al.  2004).  The  optimal  supervisory 
algorithm  has  been  synthesized  based  on  the  algorithms 
presented  in  earlier  sections. 

Figure  3  depicts  the  arrangement  of  the  message 
decoding  system,  where  each  of  the  three  processors, 
Pi,  p2  and  p3,  receives  encoded  messages  that  are  to 
be  decoded.  The  processor  p3  normally  receives  the 
most  important  messages,  and  px  receives  the  least 
important  messages.  There  is  a  server  between  each 
pair  of  processors — Si  between  px  and  p2,  s2  between 
p2  and  py  and  j3  between  p3  and  px.  Each  server  is 
connected  to  each  of  its  two  adjacent  processors  by 
a  link — the  server  Sj  is  connected  to  the  adjacent  pro¬ 
cessors  Pi  and  pk  through  the  links  Ly  and  Lkj,  respec¬ 
tively.  Out  of  these  six  links,  each  of  the  three  links, 
L\\,  L\2,  and  L2\,  is  equipped  with  a  switch  to  disable 
the  respective  connection  whenever  it  is  necessary; 
each  of  the  remaining  three  links,  L22 ,  T32,  and  L33, 
always  remain  connected.  Each  server  .q  is  equipped 
with  a  decoding  key  kt  that,  at  any  given  time,  can 
only  be  accessed  by  only  one  of  the  two  processors, 
adjacent  to  the  server,  through  the  link  connecting 
the  processor  and  the  server.  In  order  to  decode  the 
message,  the  processor  holds  the  information  on 
both  keys  of  the  servers  next  to  it,  one  at  a  time. 
After  decoding,  the  processor  simultaneously  releases 
both  keys  so  that  other  processors  may  obtain 
access  to  them. 

Figure  4  depicts  the  unsupervised  plant  model  of  the 
decoding  system  as  a  finite  state  automaton,  where 
state  1  is  the  initial  state.  The  event  py  indicates  that 
processor  pt  has  accessed  the  key  kf,  and  the  event  f 
indicates  that  the  processor  pf  has  finished  decoding 
and  (simultaneously)  released  both  keys  in  its  possession 
upon  completion  of  decoding.  The  events  f 
are  uncontrollable  because,  after  the  decoding  is 
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initiated,  there  is  no  control  on  when  a  processor  finishes 
decoding. 

Table  1  lists  the  event  cost  matrix  II.  Two  different 
control  specifications  are  investigated.  The  first  set  of 
specifications,  which  emphasizes  avoiding  deadlock,  is 
represented  by  the  7  vector  in  the  first  column  of 
table  2.  The  second  set  of  specifications,  which  focuses 
on  increasing  the  throughput  of  processor  1,  is  repre¬ 
sented  by  the  7  vector  in  the  second  column  of  table  2. 
The  positive  elements  of  the  7  vector  are  assigned  to 
the  states  8  to  16  that  represent  successful  decoding  of 
each  processor.  The  7  values  of  the  deadlock  states  26 
and  27,  where  each  processor  holds  exactly  one  key 
and  hence  no  processor  releases  its  key,  are  assigned 
negative  values.  The  remaining  states  are  non-marked 
and  are  assigned  zero  weights. 

Algorithm  5  is  applied  to  obtain  the  sequence  of 
measure  vectors  for  the  two  control  specifications. 
The  results  of  successive  iterations,  enumerating  the 
renormalized  measure  vectors,  are  presented  in  Table  3 
and  4  respectively.  The  last  column  in  each  table  is 
the  optimal  renormalized  measure  vector.  The 
optimization  requires  7  iterations  in  Case  1  and  5 
iterations  in  Case  2. 

The  optimal  configurations  for  the  plant  obtained 
under  Algorithm  5  are  depicted  in  figures  5  and  6  respec¬ 
tively.  For  supervisor  policy  1,  the  controlled  plant  is 
not  trim  and,  for  supervisor  policy  2,  there  are  discon¬ 
nected  states  in  the  controlled  model.  This  is  interpreted 
as  the  supervisor  successfully  preventing  the  plant 
from  visiting  these  states.  The  critical  values  for  the 
termination  probability  d*  computed  by  the  optimiza¬ 
tion  algorithm  for  each  control  specification  is  shown 
in  figure  7. 


Table  1.  Event  occurrence  probabilities  for 
processor  models. 


/hi 

pU 

p2\ 

p22 

p32 

p33 

/I 

/2 

/3 

0.16 

0.04 

0.16 

0.16 

0.16 

0.32 

0.00 

0.00 

0.00 

0.00 

0.16 

0.00 

0.26 

0.26 

0.32 

0.00 

0.00 

0.00 

0.37 

0.00 

0.21 

0.21 

0.21 

0.00 

0.00 

0.00 

0.00 

0.32 

0.11 

0.26 

0.00 

0.00 

0.32 

0.00 

0.00 

0.00 

0.00 

0.11 

0.00 

0.28 

0.28 

0.33 

0.00 

0.00 

0.00 

0.25 

0.00 

0.25 

0.25 

0.25 

0.00 

0.00 

0.00 

0.00 

0.28 

0.11 

0.28 

0.00 

0.00 

0.33 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.39 

0.39 

0.00 

0.22 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.14 

0.00 

0.00 

0.00 

0.43 

0.00 

0.43 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.33 

0.00 

0.33 

0.00 

0.00 

0.00 

0.00 

0.00 

0.34 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.25 

0.00 

0.00 

0.00 

0.75 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.25 

0.00 

0.00 

0.00 

0.75 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.33 

0.33 

0.34 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.33 

0.33 

0.34 

Table  2. 

Vectors  for  control  specifications. 

Case  1 

Case  2 

0.000 

0.010 

0.000 

0.000 

1.000 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.010 

0.000 

-1.000 

1.000 

0.000 

-0.200 

0.010 

0.000 

-1.000 

1.000 

0.000 

-0.200 

Next  the  stable  probability  distributions  of  the  plant 
states  are  compared  for  the  following  three  cases: 

•  Open-loop  or  unsupervised  plant 

•  Plant  with  the  optimal  supervision  policy  for 

specification  1 

•  Plant  with  the  optimal  supervision  policy  for 

specification  2 

The  distributions  are  obtained  by  considering  the  first 
row  of  the  matrix  3*,  based  on  the  measure  Vi 
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Table  3.  Iteration  vectors  for  multi-processor  model:  case  1.  Table  4.  Iteration  vectors  for  multi-processor  model:  case  2. 


Itr  1 

Itr  2 

Itr  3 

Itr  4 

Itr  5 

Itr  6 

Itr  7 

Itr  1 

Itr  2 

Itr  3 

Itr  4 

Itr  5 

-0.0616 

0.0006 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2076 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0001 

0.0063 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2074 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0055 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2101 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2167 

0.2876 

0.3232 

0.3245 

-0.0616 

0.0011 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2109 

0.2878 

0.3236 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2084 

0.2882 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0002 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2090 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2175 

0.2875 

0.3230 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2089 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0055 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2105 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0013 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2086 

0.2882 

0.3245 

0.3245 

-0.0616 

0.0010 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2078 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0011 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2114 

0.2878 

0.3235 

0.3245 

-0.0616 

0.0001 

0.0063 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2076 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0001 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2080 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0009 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2216 

0.2872 

0.3241 

0.3245 

-0.0616 

0.0000 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2147 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2116 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0014 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2084 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2105 

0.2878 

0.3232 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2110 

0.2878 

0.3236 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2077 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2077 

0.2879 

0.3245 

0.3245 

corresponding  to  state  1  which  is  the  initial  state  in  both 
cases.  If  the  stochastic  matrix  P  is  primitive  (i.e.,  irredu¬ 
cible  and  acyclic),  then  all  rows  of  -f  would  be  identical. 
However,  primitiveness  of  P  is  not  guaranteed  even  if 
the  unsupervised  plant  model  have  this  property  because 
any  subsequent  event  disabling  may  cause  loss  of 
reducibility  or  acyclic  properties. 

The  results  on  evolution  of  the  distribution  are  plotted 
in  figure  8.  While  the  unsupervised  plant  has  a  finite 
probability  of  reaching  the  deadlock  states  26  and  27, 
the  optimal  supervisors  in  both  cases  successfully 
prevent  occurrence  of  deadlock  in  the  sense  that  the 
stable  occupation  probabilities  for  states  26  and  27  are 
zero  for  each  supervisor.  However,  supervisor  2 
increases  the  throughput  of  processor  1  as  seen  from 
the  increased  probability  of  occupying  states  1  and  2. 

6.  Summary,  conclusions,  and  recommendations  for 
future  work 

This  paper  presents  the  theory,  formulation,  and 
validation  of  optimal  supervisory  control  policies  for 
dynamical  systems,  modelled  as  probabilistic  finite 


Figure  5.  Optimal  plant  configuration  for  specification  1 . 


state  automata.  The  procedure  for  synthesis  of  the  opti¬ 
mal  control  policy  relies  on  a  (renormalized)  signed 
real  measure  of  regular  languages  (Chattopadhyay  and 
Ray  2006a)  to  construct  the  performance  index.  The 
language  measure  is  based  on  the  state  transition 
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Figure  7.  Computed  9t  for  each  iteration. 


probability  matrix  of  the  underlying  finite-state  Markov 
chain  model  of  the  process  and  a  characteristic  vector  of 
state  weights,  which  serves  as  the  control  specification. 

The  main  contribution  of  the  paper  is  reformulation 
of  the  optimal  supervisor  synthesis  algorithm  (Ray 
et  al.  2004,  2005)  for  probabilistic  finite  state  plant 
models  in  terms  of  the  renormalized  measure  and 
extension  of  the  technique  to  general  non-terminating 
probabilistic  models.  Specifically,  the  work  reported  in 
this  paper  removes  a  fundamental  restriction  of  earlier 
analysis  (Ray  et  al.  2004,  Ray  2005),  namely,  each  row 
sum  of  the  state  transition  cost  matrix  II  being  strictly 
less  than  one,  instead  of  being  exactly  equal  to  one. 
The  novel  concept  of  language-based  control  synthesis, 
presented  in  this  paper,  allows  quantification  of  plant 


Figure  8.  Stable  state  probability:  unsupervised  and  super¬ 
vised  plants 


performance  instead  of  solely  relying  on  its  qualitative 
performance  (e.g.,  permissiveness),  which  is  the  current 
state  of  the  art  for  discrete  event  supervisory 
control  (Ramadge  and  Wonham  1987,  Cassandras  and 
Lafortune  1999). 

The  following  conclusion  is  drawn  in  view  of  using  the 
language  measure  for  construction  of  the  performance 
index  for  deriving  an  optimal  control  policy.  Like  any 
other  optimization  procedure,  it  is  possible  to  choose 
different  performance  indices  to  arrive  at  different 
optimal  policies  for  discrete  event  supervisory  control. 
Nevertheless,  usage  of  the  language  measure  provides 
a  systematic  procedure  for  precise  comparative  evalua¬ 
tion  of  different  supervisors  so  that  the  optimal  control 
policy(ies)  can  be  unambiguously  identified.  These 
theoretical  results  also  lay  the  foundation  for  extension 
of  the  language-measure-theoretic  framework  to  plant 
modelling  and  control,  where  all  events  may  not  be 
observable  at  the  supervisory  level. 

The  paper  provides  details  of  the  algorithms  that  are 
required  for  synthesis  of  the  optimal  supervisory  control 
policy.  These  algorithms  are  executable  in  real  time  on 
commercially  available  platforms.  Computational  com¬ 
plexity  of  the  presented  algorithms  is  polynomial  in 
the  number  of  plant  model  states.  The  concepts  are 
elucidated  with  simple  examples  and  a  relevant  engineer¬ 
ing  example.  As  such  it  is  straight-forward  to  develop 
real-time  software  codes  in  standard  languages,  based 
on  the  algorithms  provided  in  this  paper. 

There  are  several  issues  that  need  to  be  addressed  for 
implementation  of  the  theory  of  discrete-event  supervi¬ 
sory  control  in  an  operating  plant.  For  example,  the 
events  must  be  generated  in  real  time,  based  on  physical 
measurements,  to  provide  the  supervisor  with  the 
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current  information  on  the  plant;  this  is  beyond  what  is 
done  off-line  for  construction  of  the  plant  model  and 
control  synthesis.  Similarly,  the  event  disabling/enabling 
decisions  of  the  supervisor  must  be  translated  in  real 
time  as  appropriate  actions  to  control  the  plant. 


6.1  Recommendations  for  future  research 

Synthesis  of  supervisory  control  systems  may  become  a 
significant  challenge  if  some  of  the  events  are  delayed, 
intermittent,  or  not  observable  at  all,  possibly  due  to 
sensor  faults  or  malfunctions  in  network  communication 
links.  In  that  case,  the  control  algorithms  may  turn  out 
to  be  computationally  very  complex  because  of 
delayed  or  lost  information  on  the  plant  dynamics. 
Future  work  in  this  direction  should  involve  research 
on  construction  of  language  measures  under  partial 
observation  (Chattopadhyay  and  Ray  2006b)  and 
associated  synthesis  of  optimal  control  policies  to  miti¬ 
gate  the  detrimental  effects  of  loss  of  observability. 
The  latter  research  could  be  an  extension  of  the  work 
on  optimal  control  under  full  observation,  reported  in 
this  paper. 

It  would  be  a  challenging  task  to  extend  the  concept 
of  (regular)  language  measure  for  languages  higher  up 
in  the  Chomsky  hierarchy  (Flopcroft  et  al.  2001)  such 
as  context-free  and  context-sensitive  languages.  This 
extension  would  lead  to  controller  synthesis  when  the 
plant  dynamics  is  modelled  by  non-regular  languages 
such  as  the  Petri  net  (Cassandras  and  Lafortune  1999, 
Murata  1989).  The  research  thrust  should  focus 
on  retaining  the  polynomial  order  of  computational 
complexity. 

Another  critical  issue  is  how  to  extend  the  language 
measure  for  timed  automaton,  especially  if  the  events 
are  observed  with  varying  delays  at  different  states. 
Another  research  topic  that  may  also  be  worth  investi¬ 
gating  is:  how  to  extend  the  GF( 2)  field,  over  which 
the  vector  space  of  languages  is  defined  (Ray  2005),  to 
richer  fields  like  the  set  of  real  numbers. 

Areas  of  future  research  also  include  applications  of 
the  language  measure  in  anomaly  detection,  model  iden¬ 
tification,  model  order  reduction,  and  analysis  and 
synthesis  of  interfaces  between  the  continuously-varying 
and  discrete-event  spaces  in  the  language-measure 
setting.  Future  research  for  advancement  of  the  theory 
of  optimal  supervisory  control  for  discrete  event  systems 
include  the  following  areas: 

•  Robustness  of  the  control  policy  relative  to  unstruc¬ 
tured  and  structured  uncertainties  in  the  plant  model 
including  variations  in  the  language  measure 
parameters  (Lagoa  et  al.  2005) 

•  Control  synthesis  under  partial  observation  to  accom¬ 
modate  loss  of  observability  at  the  supervisory  level 


possibly  due  to  sensor  faults  or  communication  link 
failures  (Chattopadhyay  and  Ray  2006b) 

•  Construction  of  grammar-based  measures,  instead  of 
memory-less  state-based  measures  (Chattopadhyay 
and  Ray  2005),  for  non-regular  languages  when 
details  of  transitions  in  plant  dynamics  cannot  be 
captured  by  finitely  many  states 
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Appendix  A:  Derivatives  of  renormalized  measure 

This  appendix  establishes  bounds  on  the  derivatives  of 
the  renormalized  measure  v(0)  for  all  0  e  (0, 1)  and 
computes  the  limits  of  the  derivatives  as  0  — >  0+  as  an 
extension  of  what  was  reported  in  the  previous 
publication  (Chattopadhyay  and  Ray  2006a). 

The  main  result  on  boundedness  of  the  derivatives  of 
v(0)  are  presented  as  propositions.  Specifically,  the 
results  reported  in  Chattopadhyay  and  Ray  (2006a) 
are  combined  as  the  next  two  propositions. 

Proposition  A.l:  Let  4/(0)  =  0[l  —  (1  —  0)P]  *,  where  P 
is  a  ( n  x  n)  stochastic  matrix  and  n  e  N.  Then, 


(i)  Vfce  M\{1) 

..  &V(0) 

Inn - ; —  =  —k  lim 

<9^0+  d0k  0-S-O+ 


a*  -  i  4/(0) 

d0k  -  1 


[P  +  ^p-P  +  ^r1 


(ii) 


lim 

0-*O+ 


d0k 


[O-P  +  ./f1  ifk—l 

(-lfkP-P  +  ^/r1 
X  [n  -  [D  —  P  +  3>YX ]/c_1 ,  ifk  e  N\ { 1 } . 


Proof:  Given  in  Chattopadhyay  and  Ray  (2006a,  §3, 
pp.  1111-1112  as  Corollary  3  and  Corollary  6).  □ 

The  next  proposition  establishes  bounds  on  the  deriva¬ 
tives  of  v(0)  in  an  elementwise  sense  by  computing 
bounds  on  the  induced  sup-norm  of  the  derivatives  of 
4/(0).  Recall  that  /  has  been  defined  to  have  infinity 
norm  equal  to  1. 
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Proposition  A.2 


3 kv{6) 

<k\  2k+'\  inf 

[O-P  +  a^l  ‘ 

30* 

WO 

OO  \ 

V0e[O,l]. 


Proof:  Given  in  of  Chattopadhyay  and  Ray  (2006a, 
§3,  p.  1113  as  Proposition  5).  □ 

Proposition  A.3:  Denoting  the  /th  element  of  the  kXh 
derivative  of  the  measure  vector  as  (3 kv(0)/d0k)\j,  it 
follows  that 


Wk  e  {1, . . . ,  n}, 


3*v(0) 

3  0* 


3 *v(0) , 
30*  'J 


>  V0  e  [0, 1],  v{6)\  —  v(i 


Algorithm  6:  Computation  of  f? 

input  :  P,  Desired  precision  eps 

output:  & 

1  begin 

2  Set  Q[°l  =  0  ; 

3  Set  A  =  I;  /* 

4  Set  k  =  1; 

5  Set  Terminate  =  false  ; 

6  while  (Terminate  ==  false  j  do 

7  Q[t]  =  Q[*-l]  +  1  (a  -  Q[fc-1]); 

8  if  IIQW  -  Q[fc-1]ll  <  eps  then 

9  |  Terminate  ==  true  ; 

10  else 

11  k=k+ 1 ; 

12  A  =  AP; 

13  endif 

14  endw 

15  £?  =  QW; 

16  end 


/*  Zero  Matrix  */ 
Indentity  Matrix  */ 


where  n  is  the  number  of  states  in  the  plant  model. 
Proof:  First  it  is  noted  that 

OO 

p(0)  =  0^(l-0)*PAzV0e(O, 1] 

k=0 

oo 

=  0^nA(0)zV0e(O,l].  (26) 

k= 0 

Since  11(0)  is  a  matrix  of  dimension  n  x  n,  it  follows 
from  the  Cayley-Hamilton  Theorem  (Bapat  and 
Raghavan  1997)  that  integral  powers  of  11(0)  can  be 
expressed  as  polynomials  of  degree  n—  1  as  follows: 

n —  1 

Vr  e  N,  nr(0)  =  ck^k{0)  with  ck  e  C.  (27) 

k= 0 

Since  each  term  in  the  summation  on  the  left  hand  side 
of  equation  (26)  is  a  polynomial  in  0  of  degree  77  —  1 ,  it 
follows  that  the  summation  is  also  a  polynomial  in 
degree  77—1  (since  the  summation  exists  due  to  the 
sub-stochastic  property  of  11(0)).  Then  it  follows  that 
each  element  of  v(0)  is  a  polynomial  of  degree  n.  The 
result  then  follows  from  continuity.  □ 

Proposition  A.4:  For  any  stochastic  matrix  P  of 
dimension  n  x  n,  the  complexity  of  computing  the  limit¬ 
ing  matrix  ./  is  of  the  order  of  0(n3). 

Proof:  Since  the  limit  lim^oc  (1  /k)  P;  — 

always  exists,  it  is  possible  to  compute  SP  within  any 
specified  precision  simply  by  computing  the  sum 
followed  by  division  by  k,  for  a  large 
enough  value  of  k.  The  procedure  is  summarized  in 
Algorithm  6. 


Referring  to  Line  7  of  Algorithm  6,  it  is  observed  that 
is  a  stochastic  matrix  for  all  k  and  hence  it  follows 
that  the  algorithm  is  guaranteed  to  terminate  in 
( 1  /eps)  iterations,  independent  of  n.  Each  iteration 
involves  a  single  matrix  multiplication  (P  x  A)  and 
hence  algorithmic  complexity  is  of  the  same  order  as 
multiplication  of  two  n  x  n  matrices,  i.e.,  <  0(n3). 
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Abstract 

This  paper  presents  an  application  of  the  recently  developed  theory  of  language-measure-based  discrete  event  supervisory  (DES) 
control  to  aircraft  propulsion  systems.  A  two-layer  hierarchical  architecture  is  proposed  to  coordinate  the  operations  of  a  twin-engine 
propulsion  system.  The  two  engines  are  individually  controlled  to  achieve  enhanced  performance  and  reliability,  as  necessary  for  fulfilling 
the  mission  objectives.  Each  engine,  together  with  its  continuously  varying  control  system,  is  operated  at  the  lower  level  under  the 
supervision  of  a  local  discrete-event  controller  for  condition  monitoring  and  life  extension;  the  gain-scheduled  feedback  controller  that  is 
used  to  maintain  the  specified  performance  of  the  turbofan  engine  is  kept  unaltered.  A  global  DES  controller  at  the  upper  level 
coordinates  the  local  DES  controllers  for  load  balancing  and  health  management  of  the  propulsion  system. 

©  2006  Elsevier  Ltd.  All  rights  reserved. 

Keywords:  Discrete  event  supervisory  control;  Optimal  control;  Aircraft  propulsion  systems 


1.  Introduction 

Discrete-event  dynamical  behavior  of  physical  plants  is 
often  modeled  as  regular  languages  that  can  be  realized  by 
finite-state  automata  (Hopcroft,  Motwani,  &  Ullman, 
2001).  This  paper  focuses  on  development  of  intelligent 
decision  and  control  algorithms  based  on  the  theory  of 
discrete  event  supervisory  (DES)  control  (Cassandras  & 
Lafortune,  1999;  Ramadge  &  Wonham,  1987)  for  a  twin- 
engine  aircraft  propulsion  system. 

The  DES  control  system  is  designed  to  be  hierarchically 
structured  in  the  following  sense.  The  continuously  varying 
control  system  of  each  engine  interacts  with  its  own  local 
DES  controller  for  health  monitoring  and  intelligent 
control;  and  the  operational  information  is  abstracted 
and  reported  to  the  propulsion-level  DES  controller  that 
coordinates  the  operation  of  two  engines.  Furthermore,  the 
propulsion-level  supervisory  control  system  allows  interac¬ 
tions  with  exogenous  inputs,  such  as  human  operators  and 
inputs  from  other  units  (e.g.,  flight  control,  structural 
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control,  energy  management,  and  avionic  systems)  of  the 
vehicle  management  system  for  flexibility  of  making  on-line 
modifications  in  the  mission  objectives.  A  good  feature  of 
the  proposed  DES  control  system  is  that  the  supervisory 
control  policy  can  be  adaptively  updated  on-line  at  both 
engine  and  propulsion  levels  and  that  the  system  is  tolerant 
of  small  anomalies  and  component  faults. 

Although  the  theory  of  DES  control  has  been  developed 
for  almost  two  decades  (Ramadge  &  Wonham,  1987), 
only  very  few  applications  have  been  reported  in  literature. 
An  apparent  reason  is  that,  until  recently,  no  quantitative 
analytical  tool  was  available  for  design  and  evalua¬ 
tion  of  DES  controllers.  The  work  reported  in  this 
paper  makes  use  of  a  quantitative  measure  of  regular 
languages  (Ray,  2005;  Ray,  Phoha,  &  Phoha,  2005), 
and  is  a  novel  application  of  hierarchical  DES  control 
synthesis  for  the  nonlinear  complex  dynamical  system  of 
twin-engine  aircraft  propulsion.  The  real-time  implementa¬ 
tion  of  the  DES  control  scheme  is  challenging  because  it 
requires  integration  of  several  disciplines  (Kumar  &  Garg, 
1995)  such  as  systems  theory,  computer  hardware  and 
software,  and  domain  knowledge  of  gas  turbine  engine 
propulsion. 
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The  DES  control  of  the  propulsion  system  is  validated 
on  a  simulation  test  bed.  Thus,  feasibility  of  the  DES 
concept  is  demonstrated  for  enhanced  operation  and 
control  of  twin-engine  aircraft  propulsion  in  the  following 
areas:  (i)  real-time  decision-making  for  propulsion  control 
(e.g.,  load  balancing  between  the  engines),  (ii)  damage 
reduction  (with  no  significant  loss  of  performance)  via  life 
extending  control,  and  (iii)  enhanced  performance  and 
reliability  of  the  mission. 

The  terms  controller  and  supervisor  are  used  interchange¬ 
ably  in  this  paper.  The  phrases  “Upper  level”  and 
“Propulsion  level”  are  synonymous  and  so  are  the  phrases 
“Lower  level”  and  “Engine  level.”  The  paper  is  organized 
in  six  sections  including  the  present  one  and  an  appendix. 
Section  2  describes  the  real-time  simulation  test  bed  of  the 
twin-engine  propulsion  system.  Section  3  presents  the 
syntheses  of  the  engine-level  DES  control  and  propul¬ 
sion-level  DES  control  systems.  Section  4  presents  the 
simulation  results  and  discusses  implications  of  the 
controller  design.  Section  5  discusses  the  theoretical  and 
simulation  results  of  performance  evaluation  for  the 
propulsion  system  under  DES  control.  The  paper  is 
summarized  and  concluded  in  Section  6.  The  appendix 
provides  supporting  information  and  mathematical  back¬ 
ground  for  the  control  policy  synthesis  in  Sections  3  and  5. 

2.  Description  of  the  test  bed  for  propulsion  system 
simulation 

This  section  presents  implementation  of  DES  control  on 
a  real-time  simulation  test  bed  of  a  twin-engine  aircraft 
propulsion  system,  where  the  engine  model  is  similar,  in 
complexity  and  details,  to  that  reported  by  Diao  and 
Passino  (2001)  and  modular  aero  propulsion  system 
simulation  (MAPSS)  model  (Parker  &  Guo,  2002).  The 
objective  is  to  validate  the  theory  of  optimal  DES  control 
for  a  real-world  nonlinear  complex  dynamical  system  such 
as  aircraft  propulsion. 


A  DES  controlled  propulsion  system  has  been  designed 
and  implemented  on  a  simulation  test  bed  that  consists  of 
three  networked  computers  using  the  client/server  concept. 
One  of  the  three  computers  hosts  the  propulsion  system 
coordinator  for  health  monitoring  of  the  engines  and 
accordingly  making  intelligent  decisions  (e.g.,  load  balan¬ 
cing).  The  other  two  computers  run  separate  copies  (which 
may  or  may  not  be  different  depending  on  the  health  of  the 
individual  engines)  of  the  gas  turbine  engine  simulation 
model  including  its  continuously  varying  gain-scheduled 
feedback  control  system  and  a  local  discrete-event  super¬ 
visor.  The  test  bed  is  capable  of  simulating  different 
dynamics  for  individual  engines  due  to  non-uniform 
operating  conditions.  Each  of  the  engine  simulators 
integrates  the  event-driven  discrete  dynamics  modeled  by 
finite-state  automaton  as  well  as  time-driven  continuous 
dynamics  modeled  by  ordinary  differential  equations 
through  continuous-to-discrete  and  discrete-to-continuous 
interfaces  (Fu,  Yasar,  &  Ray,  2004).  Fig.  1  shows  the 
supervisory  control  architecture  of  the  engine  propulsion 
control  system.  This  software  architecture  is  flexible  to 
adapt  deterministic  finite  state  automaton  (DFSA)  models 
and  controller  designs  for  other  complex  dynamical 
systems.  Each  major  function  in  the  simulation  program 
has  a  modular  structure  as  implemented  on  the  three 
networked  computers  of  the  simulation  test  bed. 

The  C++  code  of  the  multi-layer  DES  control  system  is 
superimposed  on  the  existing  FORTRAN  simulation  code 
of  the  turbofan  engine  model  and  the  associated  con¬ 
tinuously  varying  engine  control  system  through  a  C  +  + 
program.  Specifically,  the  wrapper  program  interfaces  the 
major  inputs  and  outputs  of  the  FORTRAN  simulation 
code  with  the  rest  of  the  program  in  the  C+  +  environ¬ 
ment.  This  approach  takes  advantage  of  the  available 
FORTRAN  models  as  individual  modules  of  the  integrated 
C++  program  without  making  any  significant  changes. 

The  FORTRAN  code  of  the  turbofan  engine  simulation 
program,  which  consists  of  high-order  nonlinear  differen¬ 
tial  and  difference  equations  and  supporting  algebraic 


Fig.  1 .  Supervisory  control  architecture  of  the  propulsion  system. 
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equations,  has  been  designed  for  both  steady-state  and 
transient  operations  of  a  generic  jet  engine  (Diao  & 
Passino,  2001;  Parker  &  Guo,  2002);  and  different  control 
strategies  have  been  reported  in  the  literature  (Diao  & 
Passino,  2001;  Fu  et  al.,  2004;  Litt,  Parker,  &  Chatterjee, 
2003).  With  the  proper  inputs  such  as  power  lever  angle 
(PLA),  and  operating  condition  parameters  (e.g.,  altitude, 
forward  speed,  and  ambient  temperature),  the  FORTRAN 
engine  model  simulates  the  engine  operations  from  a  steady 
state  to  transients  and  to  (possibly)  other  steady  states. 
This  simulation  code  is  a  stand-alone  program  with  a  gain- 
scheduled  feedback  controller.  The  engine  simulation 
model  provides  various  sensor  data  (e.g.,  combustion- 
chamber  temperature  and  high-pressure  and  low-pressure 
turbine  speeds)  together  with  other  critical  information 
(e.g.,  simulation  step  size  and  simulation  cycle  number), 
which  are  collected  by  the  C++  wrapper  program  and 
exchanged  with  the  DES  controllers  through  a  typical 
message  application  protocol  interface  (API)  communica¬ 
tion  routine.  This  communication  protocol  sends  and 
receives  message  packages  through  TCP  and/or  UDP 
networks.  The  typical  delay  in  this  protocol  interface  is 
mainly  due  to  the  network  communications  and  is  found  to 
be  less  than  1  ms.  Since  the  engine  and  flight  simulations 
use  integration  step  sizes  in  the  order  of  10-20  ms,  the 
communication  delays  do  not  have  a  major  bearing  on 
performance  of  the  proposed  control  architecture. 

Fig.  2  shows  the  architecture  of  the  engine-level  plant 
and  DES  controller  implementation,  which  has  two 
replicas,  with  possible  different  parameters  and  initial 
conditions,  in  two  different  computers.  Fig.  3  shows  the 
organization  of  the  propulsion-level  DES  controller 
together  with  its  own  (discrete)  event  generator,  which  is 
implemented  on  a  third  computer  and  makes  use  of  the 


Message  API  to  communicate  with  the  other  two 
computers. 

The  DES  controller  design  has  two  important  compo¬ 
nents  that  serve  as  interfaces  between  the  continuously 
varying  control  system  and  the  discrete-event  supervisory 
controller — one  is  event  generator  and  the  other  is  action 
generator.  Event  generator  receives  continuously  varying 
sensor  data  from  the  engines.  The  data  along  with  other 
information  like  estimated  state  and  external  inputs  are 
used  to  generate  events  that,  in  turn,  are  inputs  to  the 
unsupervised  DFSA  model  of  engine  operation.  The  DFSA 
model  is  constructed  based  on  the  operation  scenario;  the 
details  are  discussed  later  in  Section  3.  The  DFSA  model 
also  serves  as  a  state  estimator  and  provides  information 
on  engine  states  and  (both  controllable  and  uncontrollable) 
events  for  the  discrete-event  supervisor  to  take  appropriate 
actions.  Event  behavior  in  the  state-based  DFSA  model  is 
dependent  on  the  state  where  the  event  is  generated  and  not 
on  the  history  or  the  path  of  how  the  state  is  reached. 

The  DES  controller  represents  the  control  policy  applied 
to  the  DFSA  model  of  engine  operation,  and  it  could  be  a 
conventional  DES  controller  based  on  the  control  specifi¬ 
cations  (Ramadge  &  Wonham,  1987)  provided  by  an 
experienced  designer;  alternatively,  an  optimal  discrete- 
event  supervisor  can  be  designed  by  the  quantitative 
method  (Fu  et  al.,  2004;  Ray,  Fu,  &  Lagoa,  2004).  In 
both  cases,  the  DES  controller  takes  the  estimated  states  as 
inputs  and  generates  control  commands  (of  controllable 
event  disabling  or  enabling)  as  outputs,  which  are 
transmitted  through  a  Message  API  communication 
routine  to  the  action  generator.  The  primary  task  of  the 
action  generator  is  to  convert  the  control  commands  from 
the  supervisor  into  necessary  input  functions  for  the 
continuously  varying  plant. 


Fig.  2.  Engine  level  plant/DES  controller  implementation. 
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Fig.  3.  Propulsion  level  DES  controller  implementation. 


3.  Synthesis  of  DES  controllers 

One  of  the  major  tasks  of  the  supervisory  decision 
making  is  fusion  of  the  (possibly)  redundant,  conflicting, 
and  incomplete  information  to  make  timely  decisions.  Such 
information  can  be  derived  from  different  types  of  sensor 
data  (Volponi,  Brotherton,  Luppold,  &  Simon,  2003)  as 
well  as  operational  history  and  the  knowledge-base 
generated  from  pilot’s  personal  experience.  Computer- 
based  advanced  analytical  techniques  are  necessary  for 
fusion  of  the  time  series  data  available  from  multiple 
sensors  and  other  relevant  non-sensor-based  information 
to  make  specific  inferences  that  could  not  be  achieved 
through  the  sole  usage  of  the  available  sensory  informa¬ 
tion.  Improved  performance  may  not  result  simply  from  an 
increased  volume  of  sensor  data  and  engine  information 
unless  the  ensemble  of  information  is  systematically 
processed  in  the  context  of  the  engine  operational 
conditions  and  mission  objectives  (Tolani,  Yasar,  Ray,  & 
Yang,  2006).  In  order  to  achieve  the  desired  performance 
of  a  DES  controller,  it  is  essential  to  have  an  effective  event 
generation  algorithm  to  ensure  fusion  of  the  heterogeneous 
information  for:  (i)  enhanced  resolution  and  reduced 
ambiguity  in  decision  and  control;  and  (ii)  advantageous 
trade-offs  between  probability  of  false  alarms  and  missed 
detection  (Basseville  &  Nikiforov,  1993). 

The  unsupervised  dynamics  of  engine  operations  are 
modeled  as  a  DFSA,  based  on  postulated  scenarios.  (Note 
that  the  model  may  change  for  different  mission  objec¬ 
tives.)  In  the  present  context,  the  DFSA  model  assumes 
that  a  twin-engine  aircraft  is  carrying  out  a  routine 
surveillance  mission.  The  mission  abortion  is  allowed  at 
certain  states  according  to  the  operation  scenario.  Each 


engine  of  the  aircraft  is  equipped  with  a  continuously 
varying  controller  which  is  supervised  by  a  local  engine- 
level  DES  controller.  The  primary  objective  of  the  local 
DES  controller  is  to  strike  the  right  balance  between  the 
conflicting  demands  of  higher  performance  from  upper 
level  supervisor  and  limiting  structural  damage  to  the 
engine  components.  The  global  propulsion-level  DES 
controller  redistributes  the  load  depending  on  the  health 
of  the  engine  and  thrust  demand  placed  by  the  pilot. 

3.1.  Engine-level  DES  control 

Fig.  4  presents  the  DFSA  model  of  the  unsupervised 
engine  operations  for  DES  control.  Table  1  lists  the  events, 
where  “C”  denotes  controllable  events  and  “UC”  denotes 
uncontrollable  events;  and  Table  2  lists  the  plant  states. 
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Table  1 

Event  list  for  the  unsupervised  engine  model 


Event 

Description 

Status 

a 

Start 

C 

b 

Warm  up  complete 

uc 

c 

Shut  the  engine 

uc 

d 

Detection  of  oscillations 

uc 

e 

Nozzle  area  reduction 

c 

r 

Engine  fails 

uc 

g 

Reduce  performance/reduce  damage 

c 

h 

Increase  performance/increase  damage 

c 

i 

Remain  in  the  state 

c 

j 

Reduce  performance 

c 

k 

Increase  performance 

c 

Table  2 

State  list  for  the  unsupervised  engine  model 

State 

Description 

Status 

qi 

Engine  start 

<12 

Engine  warm  up 

13 

High  performance/High  damage  rate 

Marked  (good) 

q4 

Oscillations 

Marked  (bad) 

q5 

Low  performance/Low  damage  rate 

Marked  (good) 

q6 

Engine  inoperable 

Marked  (bad) 

q? 

Low  performance/High  damage  rate 

Marked  (bad) 

The  events  and  states  for  the  DFSA  models  at  the  engine 
level  are  denoted  by  lower  case  letters  (e.g.,  a  is  the  start 
event  and  q,  is  the  engine  start  state).  The  engine  can 
operate  in  two  regimes,  one  is  high-performance  regime 
(state  q3),  where  the  damage  rate  is  also  high.  The  other  is 
low-performance  regime  (state  q5),  where  the  damage  rate 
is  low.  In  the  high-performance  regime,  the  engine  has  a 
tendency  to  go  to  state  q4,  where  engine  variables  like 
combustor  temperature  have  been  observed  to  have 
oscillatory  behavior.  Temperature  oscillations  could  be 
extremely  harmful  for  engine  health  (DeLaat  &  Chang, 
2003)  and  must  be  avoided  at  all  costs.  Engine-level 
controller  chooses  the  regime  of  operation  (state  q3  or  q5) 
depending  on  two  factors:  thrust  requirement  at  the 
propulsion  level  and  health  of  the  engine,  as  explained 
below. 

Health  of  the  engine  is  determined  from  the  damage 
accumulation  that  is  a  function  of  high-pressure  turbine 
gas  inlet  temperature  and  shaft  speed;  and  in  addition, 
damage  spikes  (i.e.,  sudden  jumps)  at  random  time 
intervals  are  introduced  to  simulate  the  damage  in  a  real- 
world  engine.  The  DFSA  model  of  the  supervised  engine 
operations  is  shown  in  Fig.  5,  where  the  dashed  lines 
indicate  those  controllable  events  that  are  disabled  by  the 
optimal  control  algorithm,  described  in  the  Appendix  and 
citations  therein.  The  state  q7  in  Fig.  5  becomes  unreach¬ 
able  following  the  disabling  action  of  the  supervisor. 
Therefore,  all  transitions,  originating  from  the  state  q7, 
are  also  shown  with  dashed  lines  as  well  as  the  state  itself. 


Table  3 

Event  list  for  the  propulsion  level  DFSA  model 


Event 

Description 

Status 

A 

Start  engines 

C 

B 

Warm  up  complete 

UC 

c 

One  engine  deteriorates 

uc 

D 

Redistribute  the  load  (one  engine  is  unhealthy) 

c 

E 

Bolh  engines  deteriorate 

uc 

F 

One  good  engine  fails 

uc 

G 

Bolh  engines  fail 

uc 

H 

Increase  performance 

c 

I 

Reduce  performance 

c 

J 

Request  to  abort  mission 

c 

K 

Request  accepted 

uc 

L 

Request  rejected  (both  engines  are  running) 

uc 

M 

Mission  accomplished 

uc 

N 

Turn  off  engines 

uc 

O 

Redistribute  load  (both  engines  are  unhealthy) 

c 

P 

Redistribute  load  (one  engine  has  failed) 

c 

Q 

Request  rejected  (one  engine  has  failed) 

uc 

R 

One  bad  engine  fails 

uc 

3.2.  Propulsion-level  DES  control 

One  of  the  main  tasks  of  the  propulsion-level  DES 
controller  is  to  redistribute  the  load  between  two  engines 
depending  on  the  current  health  condition  of  each  engine 
and  the  thrust  demand.  A  DES  controller  is  expected  to 
ensure  that  this  requirement  is  satisfied,  regardless  of  being 
optimal  or  not.  The  propulsion-level  DES  controller  acts  in 
an  advisory  role  for  the  mission-related  decisions  to 
enhance  the  mission  performance.  However,  the  ultimate 
decision  for  mission-related  operations  is  left  for  the  pilot. 

Since  the  propulsion-level  supervisor  is  designed  to 
execute  the  key  decisions  at  the  engine  level,  the  model 
for  operating  regimes  of  the  propulsion  system  include  the 
Cartesian  product  of  the  state  sets  of  two  (locally 
supervised)  engine  models.  However,  model  order  reduc¬ 
tion  via  aggregation  and  deletion  of  unrealizable  states  is 
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Table  4 

State  list  for  the  propulsion  level  DFSA  model 


State 

Description 

Status 

Qi 

Aircraft  on  ground 

Q2 

Engines  warming  up 

q3 

Both  engines  in  high  performance  operation 

q4 

One  engine  in  high  one  engine  in  low 
performance 

q5 

Both  engines  in  low  performance  operation 

q6 

One  engine  stopped  one  engine  in  high 
performance 

Qy 

One  engine  stopped  one  engine  in  low 
performance 

Qs 

Both  engines  failed 

Marked  (bad) 

Q9 

Decision  for  abort  mission 

Marked  (bad) 

Qio 

Mission  successful 

Marked  (good) 

Qn 

High  damage  detected  for  one  engine 

Ql2 

High  damage  detected  for  both  engines 

needed  because  the  above  Cartesian  product  may  result  in 
a  very  large  number  of  states.  The  events  and  states  of  the 
unsupervised  DFSA  model  at  the  propulsion  level  are  listed 
in  Tables  3  and  4,  respectively.  Events  and  states  for  the 
DFSA  model  at  the  propulsion  level  are  denoted  by  upper 
case  letters  (e.g.,  A  is  the  start  event  and  Q,  is  the  aircraft 
on  the  ground  state).  Multiple  occurrences  of  an  event  have 
been  indicated  as  single  events  as  seen  in  three  instances 
(i.e.,  events  D,  O  and  P)  in  Table  3.  The  event  “request  to 
abort  mission”  is  controllable  while  the  events  such  as 
“request  accepted”,  “abort  the  mission”  and  “shut  down 
the  engine”  are  uncontrollable  events  from  the  supervisor’s 
perspectives. 

4.  Simulation  experiments:  results  and  discussion 

Experiments  were  conducted  on  the  simulation  test  bed, 
described  in  Section  2,  to  validate  the  DES  control  concept. 
Upon  successful  implementation  of  the  software  modules 
on  the  client  and  server  computers,  several  simulation 
experiments  were  performed.  The  first  set  of  experiments 
was  performed  at  the  engine  level  to  demonstrate  the 
interactions  between  the  DES  controller  and  continuous¬ 
time  dynamics  of  the  engine.  The  design  specifications 
of  the  engine-level  supervisor  include  reduction  of  the 
engine  component  damage  and  consequently  engine  life 
enhancement.  Then,  the  effects  of  the  propulsion-level 
DES  controller  that  is  built  upon  the  engine-level  DES 
controllers  are  investigated. 

Fig.  6  exhibits  the  predetermined  input  profile  that 
excites  both  unsupervised  and  optimally  supervised  engine 
models.  Time  responses  of  several  outputs  (combustor 
outlet  temperature,  high-pressure  turbine  speed,  net  thrust 
of  the  engine,  and  fuel  flow  into  the  main  burner)  over  a 
period  of  12  min  were  observed.  The  four  plates  in  each  of 
Figs.  7  and  8  exhibit  the  response  profiles  of  the  above  set 
of  plant  variables  for  unsupervised  and  supervised  engine 


models,  respectively.  A  comparison  of  plots  in  Figs.  7  and  8 
indicates  that  the  optimal  DES  control  at  the  engine  level 
eliminates  the  high  frequency  oscillations  that  are  present 
in  the  unsupervised  plant  responses  in  Fig.  7.  The  super¬ 
visor  takes  actions  immediately  upon  detection  of  oscilla¬ 
tions  by  an  FFT  algorithm.  Without  making  any 
alterations  in  the  gain-scheduled  controller  of  the  engine, 
the  supervisory  actions  are  implemented  as  a  piecewise 
constant  term  in  the  reference  input  to  the  (continuous 
gain-scheduled)  controller.  In  the  continuous  control 
system,  there  are  seven  summation  points  for  the  feedback 
loop,  each  providing  a  reference  signal  for  a  specific 
actuator.  It  is  found  that  the  booster  vane  angle  and  nozzle 
area  manipulations  have  the  most  significant  effects  on  the 
response  of  engine  variables  (Tolani  et  al.,  2006).  In  the 
work  reported  in  this  paper,  nozzle  area  is  decreased  by 
20%  of  the  nominal  value.  Due  to  supervisor’s  actions,  the 
potentially  sustained  oscillations  are  quenched  in  less  than 
30  s  after  oscillations  are  observed,  and  the  engine 
operation  is  brought  to  steady  state.  Thus,  sustained 
oscillations  are  practically  non-existent  in  the  supervised 
plant  responses  in  Fig.  8.  Since  high-frequency  oscillations 
of  temperature  and  pressure  are  the  primary  sources  of 
fatigue  crack  damage  in  the  turbine  blades,  disks,  and 
stationary  vanes,  the  supervisory  control  becomes  very 
effective  for  mitigation  of  structural  damage  in  the  engine 
components.  In  contrast,  in  the  unsupervised  case,  the 
engine  health  would  be  adversely  affected  if  the  propulsion 
system  is  operated  in  this  way  to  achieve  the  mission 
objectives. 

The  propulsion-level  DES  controller  has  three  main 
tasks.  The  first  task  is  the  intelligent  decision  making  and 
control  of  the  twin-engine  aircraft  propulsion  systems  for 
mission  execution;  the  second  task  is  to  improve  the  overall 
mission  and  operational  behavior  so  that  engine  health  can 
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Fig.  7.  Simulation  output  for  the  unsupervised  case. 


be  enhanced  via  damage  reduction;  and  the  third  task  is 
load  balancing  between  two  engines  so  that  the  propulsion 
system  produces  the  thrust  demanded  by  the  flight  control 
system  while  attempting  to  enhance  engine  life.  The  issue  of 
load  balancing  becomes  even  more  important  when  the 
health  conditions  of  two  engines  are  significantly  different 
(one  can  be  in  “bad  condition”  and  the  other  in  “good 
condition”).  If  the  situation  comes  to  this  point,  then  the 
aim  of  the  DES  controller  is  judicious  redistribution  of  the 
load  between  two  engines  such  that  the  “bad  engine” 
carries  lower  load  than  the  “good  engine”,  subject  to  the 
condition  that  the  total  thrust  output  of  the  engines 
satisfies  the  mission  and  safety  requirements.  To  satisfy 
these  design  requirements,  the  propulsion-level  DES 
controller  (Kumar  &  Garg,  1995;  Ramadge  &  Wonham, 
1987),  which  may  or  may  not  be  optimal,  is  designed  and 
implemented  over  the  unsupervised  plant  model. 

Figs.  9  and  10  show  the  simulated  outputs  of  two 
engines,  where  both  engines  are  in  “good”  health  at  the 
start  of  simulation.  The  damage  increment  is  a  dimension¬ 
less  quantity  denoting  the  damage  accumulation  as 
percentage  of  the  nominal  value  at  each  simulation  cycle 


and  is  attributed  to  the  turbine  blades.  As  the  mission 
progresses,  due  to  an  injected  fault  in  Engine  2,  the  engine 
deteriorates  and  starts  to  run  at  “bad”  health  condition 
after  330  s.  After  630  s,  another  fault  is  injected  in  Engine  1 
and  this  engine  also  deteriorates.  Thus,  the  load  distribu¬ 
tion  of  the  engines  varies  in  three  regions  (see  Figs.  9  and 
10).  In  the  beginning,  the  load  is  equally  distributed  in 
Region  1.  Then  the  “good  engine”  (Engine  1)  takes  the 
responsibility  of  producing  higher  thrust  as  seen  in  Region 
2  in  the  plates  of  Figs.  9  and  10.  Later  on,  both  engines  are 
again  loaded  equally  as  seen  in  Region  3  when  it  is  not 
advisable  to  impose  uneven  thrust  requirements.  The  first 
damage  event  affects  the  performance  of  Engine  2  after 
approximately  50  s  in  order  to  realize  the  thrust  demand  of 
the  pilot.  At  this  point,  the  full  thrust  demand  cannot  be 
satisfied  if  the  load  in  Engine  2  is  reduced.  While  the  thrust 
produced  in  Region  2  by  each  engine  is  not  the  same,  the 
most  critical  impact  of  the  uneven  load  distribution  is 
formation  of  excess  moment  along  the  yaw  axis  of  the 
aircraft.  However,  it  might  be  possible  to  counteract  this 
situation  by  adjusting  the  control  surfaces  of  aircraft 
(Yasar,  Horn,  &  Ray,  2006). 
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Fig.  8.  Simulation  output  for  the  optimally  supervised  case. 


5.  Evaluations  of  DES  controllers 

The  language  measure,  described  in  Ray  (2005)  and  Ray 
et  al.  (2005),  has  been  used  to  quantitatively  evaluate 
the  impact  of  the  propulsion-level  DES  controller  on  the 
overall  mission  behavior.  Given  the  state  weights  and  the 
state  transition  probabilities,  the  language  measure  pro¬ 
vides  a  quantitative  performance  measure  of  the  control¬ 
lers.  Supporting  information  and  pertinent  mathematical 
background  are  given  in  the  Appendix. 

5.1.  Identification  of  state  transition  probabilities 

Quantitative  analysis  of  DES  controllers  and  synthesis  of 
an  optimal  DES  controller  require  identification  of  the 
state  transition  cost  matrix.  Similar  to  continuously 
varying  dynamical  systems  (CVDS),  one  must  use  the 
techniques  of  system  identification  (Ljung,  1999)  to 
evaluate  the  parameters  of  the  unsupervised  DFSA  plant 

model,  i.e.,  the  elements  ft?.  of  the  event  cost  matrix  77° 

u 

(see  Definition  2  in  the  Appendix).  As  the  number  of 
experiments  increases,  the  identified  event  costs  tend  to 


converge  within  an  appropriately  specified  tolerance.  For 
stationary  operation  of  the  engine,  since  conditional 
probabilities  of  the  events  are  assumed  to  be  time- 
invariant,  the  identified  event  costs  and  their  uncertainty 
bounds  can  be  determined.  The  probabilities  of  the  events 
such  as  deterioration  and  failure  of  an  engine  are  triggered 
by  the  event  generation  algorithm,  based  on  the  sensory 
information.  Nevertheless,  it  is  not  possible  to  simulate 
some  events,  such  as  acceptance  or  rejection  of  the  request 
by  the  pilot,  without  a  random  event  generator.  The 
randomization  used  in  triggering  this  type  of  events  has 
certainly  an  effect  on  the  identified  event  costs,  but  not 
directly,  since  sensor-based  events  are  not  induced  by  this 
randomization  (Ray  et  al.,  2005).  As  a  typical  case,  Fig.  11 
presents  identification  of  event  probabilities  at  state  Q5, 
where  both  engines  are  in  low-performance  operation. 

Fifty  simulated  missions  were  performed  for  both 
unsupervised  plant  G  and  supervised  plant  S/G  to 
construct  the  respective  state  transition  probability  ma¬ 
trices,  n°  and  77s.  The  different  visited  states  and  the 
triggered  events  were  monitored  and  plotted  to  obtain  the 
particular  77-matrices.  After  identifying  the  respective 
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Fig.  9.  Effect  of  conventional  propulsion  level  DES  controller  on  engine  1. 
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Fig.  10.  Effect  of  conventional  propulsion  level  DES  controller  on  engine  2. 


event  probability  matrices,  77°  and  77s,  of  G  and  S/G,  the 
state  transition  probability  matrices  77°  and  77s  are 
computed  using  Definition  3  in  the  Appendix.  Table  5 
lists  the  77°-matrix  of  the  unsupervised  DFSA  model  at  the 
propulsion  level. 


5.2.  Selection  of  characteristic  values  for  evaluation  of  DES 
controllers 

Given  the  state  characteristic  values  and  the  state 
transition  probabilities,  the  language  measure  serves 
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Fig.  11.  Convergence  of  event  cost  identification. 


Table  5 

n  matrix  of  the  propulsion  level  DFSA  model 


State 

Qi 

q2 

Qr 

q4 

q5 

<?6 

Qy 

q8 

Q9 

Qio 

Qn 

Ql2 

Qi 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Q2 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

q3 

0 

0 

0 

0 

0.899 

0.037 

0 

0 

0 

0 

0.051 

0.014 

q4 

0 

0 

0.500 

0 

0.500 

0 

0 

0 

0 

0 

0 

0 

q5 

0 

0 

0.772 

0 

0 

0 

0.014 

0 

0 

0.149 

0.065 

0 

Qs 

0 

0 

0 

0 

0 

0 

0.250 

0.125 

0.625 

0 

0 

0 

Qy 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

q8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Q9 

0.415 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.512 

0.073 

Qio 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Qn 

0 

0 

0 

0.044 

0.370 

0 

0 

0 

0.587 

0 

0 

0 

Ql2 

0 

0 

0 

0 

0.333 

0 

0 

0 

0.667 

0 

0 

0 

as  a 

theoretical 

performance 

measure  for 

quantitative  the 

weights 

of  the 

states  are  selected 

according  to 

evaluation  of  DES  controllers.  The  characteristic  each  state’s  importance  (contribution)  to  the  mission 

values  are  assigned  based  on  the  designer’s  percep-  management  as 

tion  for  the  importance  of  terminating  on  specific 
marked  states.  For  the  propulsion-level  DES  controller, 


x  =  [0  000000  -  1.00  -  0.188  0.300  0  0], 
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The  non-marked  states  have  no  direct  bearing  on  the 
mission  performance  and  hence  each  state  (i.e.,  states  Q[  to 
Q7,  Qn  and  Q|2)  has  been  assigned  zero  weight.  The 
marked  states,  Q8,  Q9  and  Q,0,  which  do  have  direct 
bearing  on  the  mission  performance,  are  assigned  non-zero 
weights  as  follows: 

Qs:  Both  engines  failed ,  State  Q9:  Mission  abort.  State 
Q10:  Mission  successful.  They  have  relative  weights  of  —  1, 
—0.188  and  0.3,  respectively.  The  bad  marked  state,  Q8,  is 
assigned  the  characteristic  value  of  —1.0  because  the 
aircraft  will  most  likely  be  destroyed  (because  of  both 
engines  being  non-functional)  if  the  DFSA  terminates  on 
this  state.  On  the  other  hand,  the  good  marked  state,  Q10: 
Mission  successful,  is  assigned  the  characteristic  value  of 
+0.3  based  on  its  relative  importance  to  the  loss  of  the 
aircraft.  Therefore,  by  choosing  the  characteristic  values  in 
this  way,  loosing  one  aircraft  is  made  equivalent  to 
approximately  three  successful  missions.  The  other  bad 
marked  state,  Q9:  Mission  abort,  has  also  a  negative 
characteristic  value  which  signifies  the  importance  of  this 
state  relative  to  a  successful  mission  and  a  possible  loss  of 
the  aircraft.  The  selection  of  characteristic  value  for 
mission  abort  is  to  quantitatively  match  the  theoretical 
(i.e.,  language  measure-based)  performance  of  the  unsu¬ 
pervised  plant  with  its  experiment-based  performance 
(which  will  be  introduced  in  the  next  section).  The 
corresponding  performance  measures  for  the  supervised 
cases  must  also  match  with  this  selection  provided  that  the 
plant  is  modeled  appropriately  and  the  language  measure 
parameters,  /7-matrix  and  /- vector,  are  correctly  assigned. 

Using  the  relation  ft  =  [I  —  77]  1  derived  in  Ray  (2005), 
the  language  measures  (i.e.,  the  theoretical  performance  of 
the  propulsion  system)  are  calculated  for  unsupervised 
plant  model  G  and  DES  controlled  plant  S/G  as 
Supervised  =  1  260  and  ^supervised  =  0.2055,  respectively. 

It  is  seen  that  the  DES  controller  used  in  the  propulsion 
level  has  a  positive  effect  on  the  mission  behavior  of  the 
propulsion  system. 

5.3.  Optimal  DES  controller  synthesis  and  evaluation 

After  identifying  the  event  cost  matrix  77°  of  the 
unsupervised  plant  G,  the  state  transition  cost  matrix  77° 
is  constructed  (using  Definition  3  in  the  Appendix).  State 
transition  cost  matrix  is  the  only  unknown  input  for  the 
optimal  control  algorithm  to  design  the  optimal  DES 
controller,  since  the  other  necessary  parameters,  such  as  the 
characteristic  vector  /.  are  selected  by  the  designer  based 
on  the  design  requirements.  Therefore,  given  the  state 
transition  cost  matrix  77°  of  the  unsupervised  plant  and  the 
state  characteristic  vector  /,  the  optimal  DES  controller 
can  be  synthesized  as  described  in  the  Appendix. 

Table  6  lists  the  iterations  of  optimal  control  synthesis 
for  the  propulsion-level  supervisory  control  where  the  first 
column  belongs  to  the  unsupervised  plant  G.  The 
performance  measure  of  the  unsupervised  plant  is  negative 
at  the  states  Q6,  Q7,  Qg,  Q9,  Qn,  and  Q12  as  indicated  by 


Table  6 

Iterations  for  optimal  DES  controller  synthesis 


State 

Unsupervised  plant 

Iteration  1 

Iteration  2 

Qi 

0.1260 

0.2452 

0.2452 

q7 

0.1273 

0.2477 

0.2477 

q3 

0.1286 

0.2502 

0.2502 

q4 

0.1415 

0.2617 

0.2617 

q5 

0.1572 

0.2785 

0.2785 

<?6 

-0.2555 

-0.1238 

-0.1238 

q7 

-0.1510 

0.0000 

0.0000 

Qs 

-1.0000 

-1.0000 

-1.0000 

Qq 

-0.1525 

-0.0353 

-0.0353 

Qio 

0.4248 

0.5428 

0.5428 

Qn 

-0.0250 

0.1132 

0.1132 

Ql2 

-0.0488 

0.0919 

0.0919 

Table  7 

Language  measure  and  performance  for  propulsion  level  supervisors 

Unsupervised 

Supervised 

Optimal 

Language  measure  (p) 

0.1260 

0.2055 

0.2452 

Performance  Index  (v) 

0.1276 

0.1959 

0.2480 

the  bold  scripts  in  Table  6.  All  controllable  events  leading 
to  these  states  are  disabled  and  the  resulting  performance 
measure  at  Iteration  1  shows  sign  change  at  states  Q7,  Qn, 
and  Q12  as  indicated  by  italics  in  Table  6.  All  controllable 
events  leading  to  these  states  are  now  re-enabled  for 
further  increase  in  performance  at  Iteration  2.  However, 
there  is  no  sign  change  in  the  performance  vector  between 
Iteration  1  and  Iteration  2,  which  immediately  shows 
that  the  algorithm  converged  to  the  optimal  solution 
after  this  iteration.  The  synthesis  is  complete  in  Iteration  2 
(i.e.,  there  is  no  need  to  go  for  the  Iteration  3)  because  there 
is  no  sign  change;  moreover,  the  performance  vector  at 
Iteration  2  shows  also  no  improvement  after  the  previous 
iteration. 

The  performance  of  the  optimal  controller  was  com¬ 
pared  with  that  of  unsupervised  plant  G  and  the  supervisor 
S  that  was  designed  using  the  conventional  procedure 
(Cassandras  &  Lafortune,  1999;  Ramadge  &  Wonham, 
1987).  Theoretical  performance  of  the  supervisors  can  be 
associated  with  the  language  measure  of  each  supervisor,  as 
described  in  the  Appendix.  The  language  measures  of  the 
unsupervised  plant  and  conventional  and  optimally  super¬ 
vised  plants  at  the  propulsion  level  are  listed  in  Table  7  that 
also  shows  a  close  agreement  between  the  analytically 
generated  language  measures  and  experimentally  deter¬ 
mined  values. 

Results  of  simulation  experiments  have  been  used  to 
validate  the  DES  controller  performance  based  on  the 
language-theoretic  analysis.  The  experimental  perfor¬ 
mance  index  is  determined  as  a  function  of  the  relative 
weights  of  the  visited  states.  The  mission  outcomes  of  the 
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unsupervised  and  supervised  propulsion  systems  were 
recorded  during  each  simulated  mission.  The  numbers  of 
missions  ending  at  both  good  and  bad  marked  states 
were  multiplied  by  the  respective  relative  weights  of  the 
states.  That  is,  the  experimental  performance  measure  is 
calculated  as 

..  E,=i  »'■  ■  l 

N 

where  n,  is  the  number  of  missions  ending  at  the  z'th  state, 
and  N  is  the  total  number  of  experiments;  and  Xi  is  the 
characteristic  weight  of  the  z'th  state.  The  performance  of 
the  unsupervised  plant  and  the  other  two  supervisors, 
namely,  conventionally  designed  supervisor  (Ramadge  & 
Wonham,  1987)  and  language-based  optimal  supervisor 
(Ray  et  ah,  2004),  are  compared  based  on  the  observations 
of  mission  execution  on  the  simulation  test  bed.  The 
experimental  evaluations  of  the  performance  for  different 
supervisors  are  presented  in  Table  7.  Both  theoretical  and 
experimental  (simulation)  evaluations  of  DES  controllers 
provide  better  mission  management  under  optimal  super¬ 
vision.  It  is  seen  that  the  theoretical  performance  of  the 
supervisors  is  in  quantitative  agreement  with  the  experi¬ 
mental  results,  presented  in  Table  7.  Optimal  DES 
controller,  synthesized  using  the  algorithm  described  in 
Appendix  and  in  Ray  et  al.  (2004),  has  the  highest 
theoretical  performance  (i.e.,  highest  language  measure) 
among  all  controllers.  The  results  of  the  simulation 
experiments  concur  with  the  theoretical  measure  of  the 
controllers  in  the  sense  that  optimal  supervisor  yields  the 
best  mission  performance. 

Remarks  on  the  optimal  supervisor  design :  At  the  first  step 
of  the  iterations,  the  performance  measure  of  the  un¬ 
supervised  plant  is  negative  at  states  Q6,  Q7,  Q8,  Q9,  Qn, 
and  Q12.  Therefore,  controllable  transitions  to  these  states 
should  be  disabled.  It  is  observed  that  one  engine  is  lost  at 
the  states  Q6  and  Q7,  implying  that  the  events  leading  to 
these  states  would  cause  the  “one  engine  failed”  condition 
which  is  uncontrollable  and  hence  cannot  be  disabled. 
Similarly,  the  only  transition  to  the  state  Q8  causes  the 
“both  engines  failed”  condition,  which  is  also  uncontrol¬ 
lable;  hence,  this  transition  cannot  be  disabled  either. 
The  transitions  leading  to  states  Qn  and  Q12  are  the 
“deterioration  of  engines”  condition,  which  is  directly 
related  to  the  damage  information  received  from  the 
engine-level  DES  control  system.  Based  on  this  informa¬ 
tion,  the  propulsion-level  supervisor  makes  a  decision  on 
the  health  condition  of  the  individual  engines.  Evidently, 
this  kind  of  sensory  events  are  uncontrollable,  so  is  the 
engine  deterioration  event.  The  remaining  state  that  should 
be  investigated  is  the  state  Q9,  Mission  abort.  The  optimal 
control  algorithm  disables  all  mission  abort  requests,  which 
significantly  increases  the  performance  of  the  mission 
behavior  with  increased  risk  of  losing  the  aircraft.  The 
simulation  experiments  show  that  the  mission  performance 
at  the  propulsion  level  increases  under  the  optimal  DES 
controller,  albeit  with  an  increased  probability  of  aircraft 


loss.  Table  7  shows  a  close  agreement  between  the 
analytically  generated  language  measures  and  experimen¬ 
tally  determined  performance  data.  It  should  be  noted  that, 
the  optimal  control  policy  is  likely  to  change  if  the  elements 
(i.e.,  individual  state  weights)  of  the  j(-vector  are  altered. 

6.  Summary  and  conclusions 

This  paper  presents  a  quantitative  approach  to  analysis 
and  synthesis  of  hierarchical  DES  control  laws  for  aircraft 
propulsion  systems.  The  objectives  are: 

•  Intelligent  decision  and  control  of  distributed  propul¬ 
sion  management  systems,  where  each  of  the  engines  has 
its  own  local  DES  control. 

•  Structural  damage  reduction  and  life  extension  of 
aircraft  engines  without  any  significant  loss  of  the 
system  performance. 

•  Decision  making  and  mission  planning  modifications 
through  a  high-level  DES  coordinator. 

•  Incorporation  of  optimal  control  laws  for  enhanced 
mission  management. 

•  Extension  of  this  work  to  other  complex  dynamic 
systems  such  as  rotorcrafts  and  power  plants  simulation 
test  beds. 

A  decision  and  control  architecture  has  been  proposed  to 
coordinate  the  operations  of  a  twin-engine  propulsion 
system.  The  DES  control  law  has  been  validated  for  a  twin- 
engine  aircraft  propulsion  system  on  a  networked  simula¬ 
tion  test  bed.  The  plant  dynamics  in  the  simulation  test  bed 
is  built  upon  the  model  of  a  generic  turbofan  gas  turbine 
engine.  The  software  architecture  of  the  simulation  test  bed 
is  flexible  for  adaptation  to  arbitrary  DFSA  models  and  a 
variety  of  DES  control  laws,  including  those  that  are 
quantitatively  analyzed  using  a  language  measure. 
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Appendix  Language  measure  and  discrete  event  optimal 
control 

This  appendix  reviews  the  previous  work  on  language 
measure  (Ray,  2005;  Ray  et  al.,  2005)  and  optimal  control 
policy  (Ray  et  al.,  2004)  that  is  based  on  this  measure  with 
no  event  disabling  penalty.  The  background  information 
necessary  to  develop  a  performance  index  for  the  optimal 
DES  control  law  is  introduced.  Performance  of  DES 
controllers  is  associated  with  the  language  measure 
through  out  this  paper. 
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Let  the  dynamical  behavior  of  a  physical  plant  be 
modeled  as  a  deterministic  finite  state  automaton  (DFSA) 
Gj  =  (Q, X, 8,qh  Qni),  where  Q  is  the  finite  set  of  states  qj 
with  |  Q\  —  n  and  qf  e  Q  is  the  initial  state;  X  is  the  (finite) 
alphabet  of  events  with  |T|  =  m:  the  function  <5  :  Q  x  X  -» 
Q  represents  state  transitions  and  Qm  c  Q  is  the  set  of 
marked  states  which  have  some  importance  (positive  or 
negative)  for  the  DFSA  model. 

Definition  1.  The  characteristic  state  weight  function  that 
assigns  a  signed  real  weight  to  states  is  defined  as:  y  :  Q  —¥■ 
[—  1,  1]  such  that 

r  [—1,  0)  if  qjeQ-, 

Zj  -  7<.cij)  e  \  f°l  'UjtQm, 

(  (0,  1]  if  qj  e  Q+, 

where  Q~  c  Q  and  (T,  c  Q  are  the  set  of  negatively  and 
positively  marked  states  respectively.  The  (n  x  1)  charac¬ 
teristic  vector  is  denoted  as 

i  =  [zi  h  ■  ■  •  x„]T 

Definition  2.  The  event  cost  is  the  relative  frequency  of 
occurrence  of  an  event  given  the  DFSA  state  at  which  the 
event  is  generated,  and  is  defined  as 

•  n[(Jk  |  qj\  =  ftjk  e  [0,  1),  relative  frequency  of  occurrence 
of  event  k  at  state  j; 

•  n[(Jk\qj\  =  0  if  Siqj^k)  is  undefined,  i.e.,  event  k  is  not 
defined  at  state  j. 

The  (n  x  m)  event  cost  matrix  is  denoted  as  77  =  [ft,y]. 

Definition  3.  The  state  transition  cost  of  the  DFSA  is  a 
function  7t :  Q  x  Q  *  [0, 1)  defined  as  the  relative  fre¬ 
quency  of  transition  from  state  j  to  state  k  such  that 
n(qk\qj) —Y^k{a\qj)  =  njk  and  the  nxn  state  transition 
cost  matrix,  denoted  as  77-matrix,  is  defined  as 


7ln 

n\2  ■  ■ 

7^1  n 

7t21 

7122  ‘  ‘ 

^2  n 

R-ni 

Note  that  event  costs  and  state  transition  costs  are  very 
much  similar  to  probabilities  and  probabilistic  interpreta¬ 
tion  is  given  in  Ray  et  al.  (2005).  In  this  sense,  an  event  cost 
can  be  analogous  to  the  probability  of  an  event  to  occur  at 
a  state,  and  a  state  transition  cost  can  be  analogous  to  the 
probability  of  leaping  from  one  state  to  another. 

Now  we  define  the  language  measure  in  terms  of  the 
signed  state  weight  function  y  and  the  non-negative  state 
transition  cost  %. 

Definition  4.  The  signed  real  measure  of  the  language  L(G,) 
created  by  a  DFSA  G,,  initialized  at  the  state  <y,  e  Q.  is 
defined  as 

h,  =  /i(7(G,)). 


The  (n  x  1)  real  signed  measure  vector  is  denoted  as 

fi  =  [if  k 2  ■■■  h»]T- 

It  has  been  shown  in  Ray  (2005),  and  Ray  et  al.  (2005) 
that  the  measure  of  the  language  L(G,),  where  G,  = 
(Q,  X,  3,  qt,  Qm)  can  be  expressed  as  ^  kj  +  Zi- 

Equivalently,  in  vector  notation:  fi  —  lip  +  /.  Therefore, 
the  measure  vector  fi  is  uniquely  determined  as 

fi  =  [i-  n]~li 

In  the  gas  turbine  engine  application,  the  penalty  of 
disabling  controllable  events  (e.g.,  redistribution  of  thrust 
between  two  engines,  and  nozzle  area  reduction  for 
individual  engines)  is  set  to  zero  as  these  manipulations 
do  not  require  any  special  effort. 

The  state-based  optimal  control  policy  is  obtained  by 
selectively  disabling  controllable  events  to  maximize  the 
measure  of  the  controlled  plant  language.  In  each  iteration, 
the  optimal  control  algorithm  attempts  to  disable  all 
controllable  events  leading  to  “negatively  marked  states” 
and  enable  all  controllable  events  leading  to  “positively 
marked  states”.  It  has  been  also  shown  in  Ray  et  al.  (2004, 
2005)  that  computational  complexity  of  the  control 
synthesis  is  polynomial  in  the  number  of  plant  states. 

The  algorithm  for  synthesis  of  the  optimal  control  policy 
is  summarized  as  follows:  let  G  be  the  DFSA  plant  model 
without  any  constraint  of  operational  specifications.  Let 
the  state  transition  cost  matrix  of  the  unsupervised  plant 
be:  77 p,ant  e  SR'!X"  and  the  characteristic  vector  be: 
X  =  [yA  y2  ■  •  ■  x(,]T-  Starting  with  iteration  index  k  —  0, 
and  77°  =  nplant,  the  control  policy  is  constructed  by  the 
following  two-step  procedure: 

Step  1:  For  every  state  q for  which  )/■  <  0,  disable 
controllable  events  leading  to  qj.  Now,  771  =  77°  —  d°, 
where  A0  >  0  is  composed  of  event  costs  corresponding  to 
all  controllable  events  that  have  been  disabled  at  k  =  0. 

Step  2:  Starting  with  k  —  1,  if  plj  >  0,  re-enable  all 
controllable  events  leading  to  qj ,  which  were  disabled  in 
Step  1.  The  cost  matrix  is  updated  as:  77/f+l  =  77A  +  Ak  for 
k  >  1,  where  Ak  >  0  is  composed  of  event  costs  corre¬ 
sponding  to  all  currently  re-enabled  controllable  events. 
The  iteration  is  terminated  when  no  controllable  event 
leading  to  qj  remains  disabled  for  which  fik  >  0,  i.e.,  if  there 
is  no  sign  change  in  the  measure  vector  fi  between  two 
consecutive  iteration  steps.  At  this  stage,  the  optimal  value 
of  the  performance  index,  which  is  the  language  measure,  is 
f*  =  [I  —  77*]— 1 X- 
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